#2018-03-1215:17ghadiour depstar tool just got an awesome logo courtesy @quephird#2018-03-1215:18ghadihttps://raw.githubusercontent.com/healthfinch/depstar/master/depstar_logo.png#2018-03-1215:18alexmillerI’ll pretend I didn’t see that :)#2018-03-1215:21alexmillerdoes quephird work with you?#2018-03-1215:21ghadiyes @alexmiller#2018-03-1215:21alexmilleraw, cool. she’s great.#2018-03-1215:22alexmiller;)#2018-03-1215:22alexmillerI meant to put all that in pm to Ghadi :)#2018-03-1215:41devn🙂#2018-03-1217:39alexmillerAdded a wiki page with tools that use t.d.a or clj https://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-03-1217:39alexmillerlinked from the readme too if you’re trying to find it later#2018-03-1217:54seancorfieldI think that last line should be cljs-test-runner, not clj-test-runner (link is correct, text is not).#2018-03-1217:55alexmillerthx#2018-03-1218:33ghadidepstar is one word @alexmiller - thx for the mention#2018-03-1218:34alexmillerit’s a wiki, fix as you like :)#2018-03-1218:34alexmillerand if it’s misconfigured for edits, let me know#2018-03-1218:57ghadiI'm getting a 403 when I clone then push @alexmiller#2018-03-1218:58thheller@alexmiller https://github.com/thheller/shadow-cljs also supports using deps.edn via clojure. can't edit wiki.#2018-03-1218:58ghadi(I can't remember if there is supposed to be an edit button in the UI -- but there isn't one visible)#2018-03-1219:00alexmillerI’m sure it’s set wrong - seems like the default is always wrong :)#2018-03-1219:00alexmiller@ghadi clone then push what?#2018-03-1219:01ghadithe wiki repo#2018-03-1219:01ghadiI think I was supposed to edit in the UI#2018-03-1219:05alexmillerwell that’s how normal people do it :) I’m getting Stu to change the setting#2018-03-1221:21dominicm@alexmiller going on the current line of thinking, it would be useful if https://github.com/clojure/tools.deps.alpha/blob/88f8e4f0113ba555c3c667bd5fbf60f3887f8140/src/main/clojure/clojure/tools/deps/alpha/reader.clj#L18-L28 let me specify other options particularly -Srepro#2018-03-1221:27alexmillerother options like?#2018-03-1221:28alexmilleroh, I see what you mean#2018-03-1221:28alexmillergood idea, feel free to jira / patch#2018-03-1221:28alexmillerthat affects memoization of course so would need to probably expose the scrape function with a better name#2018-03-1221:34dominicmtbh, if I was being very specific, it might be just that I need the version that Stu originally created where the different config files get labelled.#2018-03-1221:36alexmillerwell that idea is still in play for several reasons#2018-03-1221:37alexmillerprob the same reasons you’re interested in it#2018-03-1221:37alexmillerthat’s going to be a mildly painful change though across the script and tda#2018-03-1221:44dominicmalpha, baby 🙂#2018-03-1221:51dominicm(I can picture Austin powers saying that)#2018-03-1222:07mhuebertcan a github dep in deps.edn specify the root path (into the repo) for the desired library?#2018-03-1222:08mhuebertI have been publishing more than one library from a single repo#2018-03-1222:08alexmillerYeah with :deps/root#2018-03-1222:08mhuebertwould that be a relative or absolute path?#2018-03-1222:08mhuebert/my-lib#2018-03-1222:09alexmillerRelative in the repo#2018-03-1222:09alexmillerTo the base dir#2018-03-1222:12dominicm@mhuebert fyi, when a transitive dependency does :local/root, it is relative to where you're running clj from. So you can't put things in sub-dirs, it has to be flat.#2018-03-1222:12dominicmWe're doing this, you're not alone it's good fun 🙂#2018-03-1222:14mhueberthmm. ok. i am not using :local/root anywhere at the moment#2018-03-1222:14mhuebertbut I am seeing:
Error building classpath. nil
java.lang.NullPointerException
at clojure.string$starts_with_QMARK_.invokeStatic(string.clj:364)
at clojure.string$starts_with_QMARK_.invoke(string.clj:360)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:48)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:43)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_fetch.invokeStatic(impl.clj:64)
at clojure.tools.gitlibs.impl$git_fetch.invoke(impl.clj:61)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
#2018-03-1222:14mhuebertfor the dep
lark/tools {:git/url ""
:sha "cc1305534b1371df3962fd46503dd0478d81a08a"
:deps/root "tools"}
#2018-03-1222:15mhuebertI can run deps from that directory locally (“tools”), it has a deps.edn file#2018-03-1222:15dominicmfwiw, that should work, I used it at some point, 100%#2018-03-1222:15mhueberthmm.#2018-03-1222:18mhuebert#2018-03-1222:18seancorfield@mhuebert FWIW, I just ran (! 581)-> clj -Sdeps '{:deps {lark/tools {:git/url ""
:sha "cc1305534b1371df3962fd46503dd0478d81a08a"
:deps/root "tools"}}}
'
and it worked fine and gave me a REPL.#2018-03-1222:18mhueberthmm#2018-03-1222:19mhuebertok thanks for trying that. maybe i need to update my clj#2018-03-1222:21mhuebertwell that’s weird. that exact command doesn’t work on my machine, with latest clj from brew#2018-03-1222:24seancorfieldI'm on the latest clojure 1.9.0.358 already installed#2018-03-1222:24seancorfield(output from brew upgrade clojure)#2018-03-1222:24mhuebertyeah, me too#2018-03-1222:25seancorfieldI created a new folder mhuebert and dropped into there to run that command with no deps.edn file present, just to confirm it works on its own.#2018-03-1222:25seancorfieldIf you have deps.edn, it will read that and maybe the NPE is coming from something else there?#2018-03-1222:26mhuebertyeah I tried the same thing, ran it from a fresh directory in a new terminal window#2018-03-1222:26mhuebertmysteries of the universe#2018-03-1222:26seancorfieldMaybe something in your ~/.clojure/deps.edn then?#2018-03-1222:27mhuebertappears to be all commented out#2018-03-1222:29mfikesTry -Srepro to be sure#2018-03-1222:30mhuebertthat produces a repl#2018-03-1222:30mfikesYou must have something in your deps.edn chain 🙂#2018-03-1222:30mhuebertsomehow the url passed to call-with-auth is nil, when it’s trying to fetch from github#2018-03-1222:31mfikesOther hunch: Maybe a cache gone bad? and it needs to be forced#2018-03-1222:31mhuebert(defn- call-with-auth
([^GitCommand command]
(call-with-auth
(.. command getRepository getConfig (getString "remote" "origin" "url"))
command))
([^String url ^GitCommand command]
(if (and (instance? TransportCommand command)
(not (str/starts-with? url "http")))
(.. ^TransportCommand command (setTransportConfigCallback @ssh-callback) call)
(.call command))))
#2018-03-1222:31mhuebertthird from bottom, the str/starts-with?#2018-03-1222:31mhuebertis there some caching somewhere that I can clear?#2018-03-1222:32mfikesI''d try without -Srepro and add -Sforce (but I'm speculating)#2018-03-1222:34mhuebertoh, I maybe misread/communicated, I tried -Srepro by itself earlier#2018-03-1222:34mhuebertclj -Srepro -Sforce -Sdeps '{:deps {lark/tools {:git/url ""
:sha "cc1305534b1371df3962fd46503dd0478d81a08a"
:deps/root "tools"}}}'
#2018-03-1222:34mhuebertstill null pointer. hmm.#2018-03-1222:39mhueberti wonder if it could have anything to do with git auth#2018-03-1222:41mhuebertwell this will have to wait for another day#2018-03-1307:31mhueberttrying clojure.tools.gitlibs, I get a failure loading slf4j:
user=> (require '[clojure.tools.gitlibs :as gl])
nil
user=> (gl/resolve "" "cc1305534b1371df3962fd46503dd0478d81a08a")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
NullPointerException clojure.string/starts-with? (string.clj:364)
#2018-03-1307:33mhuebertadding slf4j-simple to class path resolves that warning but doesn’t affect the error#2018-03-1307:41mhuebertok, I removed the braintripping directories from ~/.gitlibs/_repos (inside <mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection> and https://github.com) and now it works#2018-03-1307:41mhuebertmaybe there was some corrupt thing there#2018-03-1307:42mhuebertunfortunately I can’t find a way to reproduce this now#2018-03-1307:43mhuebertoh, now i’ve got it. found the old dir in the trash.#2018-03-1307:44mhuebertwell, that is simple.#2018-03-1307:46mhuebertReproduction steps:
1. create an empty directory at ~/.gitlibs/_repos/github.com/<user>/<repo>
2. try loading any version of *main-repl-fn*#2018-03-1320:45bhaumanthen we could set that with an -e "(set! clojure.main/*main-repl-fn* rebel-readline.main/repl)"#2018-03-1320:46bhaumanand then bobs your uncle#2018-03-1320:48alexmillerclojure.main is in Clojure, so this would require a change in Clojure 1.10. I think ultimately we’ll want a change that does something like that. But I’d like to look at a shorter range version.#2018-03-1320:50dominicmI feel like I'm missing some context?#2018-03-1320:54bhauman@dominicm I was just trying to think of a lightweight way for clj to inject rebel-readline, adding an inline dep and extra e arg would be easy#2018-03-1320:55seancorfieldCurrently, I have :rebel {:extra-deps {com.bhauman/rebel-readline {:mvn/version "RELEASE"}}
:main-opts ["-m" "rebel-readline.main"]}
in my ~/.clojure/deps.edn file and just say clj -A:rebel when I want a rebel REPL...#2018-03-1320:55dominicmyeah, I have that, but I namespaced it as :user/rebel.#2018-03-1320:55bhaumanbut that doesn't help clj#2018-03-1320:56dominicmno?#2018-03-1320:56dominicmOH. You mean for including rebel in clj?#2018-03-1320:56bhaumanyes 🙂#2018-03-1320:56dominicm🤔 does rebel have any dependencies#2018-03-1320:57bhaumanextremely flat, tools reader is the only one I still want to get rid of#2018-03-1320:57bhaumanthat requires me writing my own indenting code#2018-03-1320:58bhaumanafter that there would be no deps besides Jline of course#2018-03-1320:58dominicmI mean, you could use Mr. Anderson to package it?#2018-03-1321:00bhaumanoh interesting, didn't know about that#2018-03-1321:08dominicmShading is a pretty cool thing 🙂#2018-03-1322:15dominicm@seancorfield do you have your handy snippet for clj + socket server?#2018-03-1322:16seancorfieldThis one? :socket {:jvm-opts ["-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl}"]}
#2018-03-1322:19dominicmYep#2018-03-1322:20dominicmclj -J-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl} -A:dev:build is giving me
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
at clojure.lang.RT.seqFrom(RT.java:547)
at clojure.lang.RT.seq(RT.java:527)
at clojure.lang.APersistentMap.cons(APersistentMap.java:40)
at clojure.lang.RT.conj(RT.java:667)
at clojure.core$conj__4932.invokeStatic(core.clj:85)
at clojure.core$merge$fn__5498.invoke(core.clj:3029)
at clojure.core$reduce1.invokeStatic(core.clj:936)
at clojure.core$reduce1.invokeStatic(core.clj:926)
at clojure.core$merge.invokeStatic(core.clj:3028)
at clojure.core.server$parse_props$fn__8321.invoke(server.clj:149)
at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
at clojure.core.protocols$fn__7665.invokeStatic(protocols.clj:168)
at clojure.core.protocols$fn__7665.invoke(protocols.clj:124)
at clojure.core.protocols$fn__7620$G__7615__7629.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__7642.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__7642.invoke(protocols.clj:75)
at clojure.core.protocols$fn__7594$G__7589__7607.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6704)
at clojure.core.server$parse_props.invokeStatic(server.clj:143)
at clojure.core.server$start_servers.invokeStatic(server.clj:154)
at clojure.core.server$start_servers.invoke(server.clj:154)
at clojure.lang.Var.invoke(Var.java:381)
at clojure.lang.RT.doInit(RT.java:486)
at clojure.lang.RT.<clinit>(RT.java:333)
... 1 more
java.lang.NoClassDefFoundError: Could not initialize class clojure.lang.RT
at clojure.lang.PersistentQueue.pop(PersistentQueue.java:113)
at clojure.lang.PersistentQueue.pop(PersistentQueue.java:25)
at clojure.lang.Agent$Action.doRun(Agent.java:149)
at clojure.lang.Agent$Action.run(Agent.java:163)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
which is interesting#2018-03-1322:21dominicmWorks in ~/.clojure/deps.edn though.#2018-03-1322:21seancorfieldYou need "..." around it.#2018-03-1322:22seancorfieldclj -J"-Dclojure..."#2018-03-1322:22seancorfieldOr this clj -J-Dclojure.server.repl="{:port,5555,:accept,clojure.core.server/repl}"
#2018-03-1322:24dominicmI did the latter originally. But I got the above error.#2018-03-1322:32seancorfield(! 614)-> clj -J-Dclojure.server.repl="{:port,5555,:accept,clojure.core.server/repl}"
Clojure 1.9.0
user=> ^D
Tue Mar 13 15:32:30
(sean)-(jobs:0)-(~/clojure/lsnape)
(! 615)-> clj -J-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl}
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
#2018-03-1322:33seancorfieldI get the exception without the quotes -- but it works with quotes as seen above.#2018-03-1322:39dominicmI'll explore it later. ¯\(ツ)/¯#2018-03-1403:51jebberjebI’ve just been doing $> JAVA_OPTS="-Dclojure..." clj. Same for lein#2018-03-1403:53jebberjebMy thinking was that it’s nice because it doesn’t depend on how a particular thing handles command line args?#2018-03-1407:50dominicmI didn't know that worked for clj, neat.#2018-03-1411:52niclasnilssonI want to get these types of things working, and I think I’ve tried quoting and escaping the things after clj in all ways possible, but I bet there is a way I haven’t tried, since I can’t get it to work:
#!/usr/bin/env clj -Sdeps '{:deps {clj-time {:mvn/version "0.14.2"}}}'
(require '[clj-time.core :as t])
(println (str "Time is now " (t/now)))
I either get a "Error building classpath. Don't know how to create ISeq from: clojure.lang.Symbol" or a "Error while parsing option "--config-data {:deps": java.lang.RuntimeException: EOF while reading" for a combinations that I’ve tried. I bet someone knows how to do this properly?#2018-03-1412:15dominicm@niclasnilsson what OS are you on? That won't work for Linux.#2018-03-1412:47niclasnilsson@dominicm MacOS#2018-03-1412:48niclasnilssonDo you mean like “there is no way to make it work”?#2018-03-1412:51dominicmThere's a limitation on how shebangs are read. It doesn't apply to mac though.
You may find it works if you don't use env #2018-03-1413:12niclasnilssonAh, ok. Will try again then. Thanks!#2018-03-1416:19seancorfieldAlso: try using , instead of space in the deps string. #2018-03-1416:30dominicm@seancorfield Yes, I think you might have the right of it there. ' doesn't have have the same effect in shebangs as on the shell.#2018-03-1417:09seancorfieldConfirmed that the following works @niclasnilsson (! 622)-> ./time.sh
Time is now 2018-03-14T17:09:15.652Z
Wed Mar 14 10:09:15
(sean)-(jobs:0)-(~/clojure/bin)
(! 623)-> cat time.sh
#!/usr/bin/env clj -Sdeps {:deps,{clj-time,{:mvn/version,"0.14.2"}}}
(require '[clj-time.core :as t])
(println (str "Time is now " (t/now)))
On macOS#2018-03-1417:22hiredmanfootnote to "On macOS" https://unix.stackexchange.com/questions/63979/shebang-line-with-usr-bin-env-command-argument-fails-on-linux#2018-03-1419:56dominicmI didn't know BSD did it right too.#2018-03-1417:11niclasnilssonAh… I wasn’t even close to try that combination. Neat, thanks, @seancorfield!#2018-03-1417:12niclasnilssonWill try myself later tonight, when I get back to this.#2018-03-1506:20niclasnilssonWorked fine! Thanks, @seancorfield#2018-03-1512:16delaguardoIs it possible to redefine local maven repo location with tools-deps?#2018-03-1512:21alexmillerYes, I think the key is :mvn/local-repo#2018-03-1512:21alexmillerAt the root of deps.edn#2018-03-1512:23alexmillerYeah, that’s it#2018-03-1512:24delaguardoCool! thanks. will check this out#2018-03-1512:37mpenetany eta for TDEPS-9, or if another approach is considered for that issue?#2018-03-1512:44mpenethaving a blank: what's the way to get a dependency from a subdir in a git repo with multiple dependencies ?#2018-03-1512:44mpenetI think I got that answered a while back, but I can't remember the answer (and well... slack history is not helping)#2018-03-1512:51niclasnilssonSometimes I wonder how Slack managed to take over the world so much. The history problem sucks for large open things like Clojurians.#2018-03-1514:24seancorfieldOff-road topic for this channel -- take it to #community-development if you want to discuss Slack/history/etc. #2018-03-1512:47mpenetit might be :deps/root reading the source#2018-03-1512:51alexmilleryes#2018-03-1512:55alexmilleron TDEPS-9, no eta. I’m consulting right now so only getting a few hrs a week on tdeps. The windows installer is my top priority at the moment, hoping to get that done this week.#2018-03-1514:25delaguardoI found strange bug -
15:24 $ clj -Srepro -Sdeps "{:mvn/local-repo \".m2/repository\"}" -Spom
Downloading: org/clojure/clojure/1.9.0/clojure-1.9.0.pom from
Downloading: org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.pom from
Downloading: org/clojure/pom.contrib/0.2.2/pom.contrib-0.2.2.pom from
Downloading: org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.pom from
Downloading: org/clojure/clojure/1.9.0/clojure-1.9.0.jar from
Downloading: org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar from
Downloading: org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar from
Unknown option: "--config-data"
#2018-03-1514:26delaguardoIt happens only when I’m trying to regenerate pom.xml file with local-repo#2018-03-1514:28alexmillerthe local-repo is an absolute path btw, although I don’t think that’s the issue you’re seeing#2018-03-1514:29delaguardobut pom.xml is not generated in that case#2018-03-1514:29alexmillerif you can file a jira in https://dev.clojure.org/jira/browse/TDEPS that would be helpful. I can repro locally and I suspect I know where the issue is#2018-03-1514:29delaguardoWill do#2018-03-1514:32alexmillerthere’s some duplicated code in the make classpath and pom gen stuff and I don’t think the latter was updated the last time the former changed#2018-03-1515:23seancorfieldFWIW, that command ultimately becomes this + /usr/bin/java -Xmx256m -classpath /usr/local/Cellar/clojure/1.9.0.358/libexec/clojure-tools-1.9.0.358.jar clojure.main -m clojure.tools.deps.alpha.script.generate-manifest --config-files /usr/local/Cellar/clojure/1.9.0.358/deps.edn,deps.edn --gen=pom --config-data '{:mvn/local-repo ".m2/repository"}'
and that's what complains about --config-data#2018-03-1515:25alexmilleryeah, I’m aware - the generate-manifest hasn’t tracked changes in make-classpath#2018-03-1517:42mhueberthas anyone seen something like this when running -Spom?
clj -Spom
Error generating pom manifest: called children on a leaf node
java.lang.Exception: called children on a leaf node
at clojure.zip$children.invokeStatic(zip.clj:80)
at clojure.zip$append_child.invokeStatic(zip.clj:223)
at clojure.zip$append_child.invoke(zip.clj:223)
at clojure.tools.deps.alpha.gen.pom$xml_update$fn__2247.invoke(pom.clj:94)
at clojure.tools.deps.alpha.gen.pom$xml_update.invokeStatic(pom.clj:87)
at clojure.tools.deps.alpha.gen.pom$xml_update.invoke(pom.clj:83)
at clojure.tools.deps.alpha.gen.pom$replace_deps.invokeStatic(pom.clj:98)
at clojure.tools.deps.alpha.gen.pom$replace_deps.invoke(pom.clj:96)
at clojure.tools.deps.alpha.gen.pom$sync_pom$fn__2265.invoke(pom.clj:120)
at clojure.tools.deps.alpha.gen.pom$sync_pom.invokeStatic(pom.clj:117)
at clojure.tools.deps.alpha.gen.pom$sync_pom.invoke(pom.clj:112)
at clojure.tools.deps.alpha.script.generate_manifest$_main.invokeStatic(generate_manifest.clj:49)
at clojure.tools.deps.alpha.script.generate_manifest$_main.doInvoke(generate_manifest.clj:27)
#2018-03-1517:47alexmillerwhat’s your existing pom.xml?#2018-03-1517:48alexmillerdoes it have a <dependencies>?#2018-03-1517:54mhuebertlet me see#2018-03-1517:55mhuebertyes it does#2018-03-1517:56mhuebertI can run -Spom if I remove the existing pom.xml#2018-03-1518:20alexmillerwell if you can capture the case when it fails, would be happy to take a ticket for that#2018-03-1518:21alexmillerthe pom.xml when it fails that is#2018-03-1520:20mhueberthere is a minimal pom.xml which produces the error:
<?xml version="1.0" encoding="UTF-8"?>
<!-- generated by shadow-cljs pom, do not edit -->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>editor</groupId>
<artifactId>editor</artifactId>
<version>0.0.1</version>
<name>editor</name>
<dependencies>
<dependency>
<groupId>thheller</groupId>
<artifactId>shadow-cljs</artifactId>
<version>2.2.9</version>
</dependency>
</dependencies>
</project>
#2018-03-1520:20mhuebertI can create a ticket for that if it would be helpful#2018-03-1520:21alexmilleryes, also would help to know which jdk you’re using#2018-03-1520:22alexmillerjava -version#2018-03-1520:23mhuebertjava version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
#2018-03-1520:27mhueberthttps://dev.clojure.org/jira/browse/TDEPS-55#2018-03-1520:27alexmillerI also assume you’re using latest clj (`clj -Sverbose`) ?#2018-03-1520:28alexmillerlatest is 1.9.0.358#2018-03-1520:54mhuebertyes#2018-03-1520:55alexmillerk, if I get a chance I’ll look at it tomorrow#2018-03-1520:55alexmillerthis doesn’t seem much different than everything else I’ve done that works fine so not sure what the deal is#2018-03-1521:07seancorfieldLooks like it's the comment that causes problems @mhuebert /cc @alexmiller#2018-03-1521:07alexmillerI was just about to say that :)#2018-03-1521:08alexmillerthere’s an assumption in the code about the doc structure that’s being foiled there#2018-03-1521:08seancorfieldIf you move the comment inside <project> it's fine. Or delete it.#2018-03-1521:09seancorfield(sorry, it made me too curious to resist trying to figure out what was causing it)#2018-03-1521:09alexmillerI will fix it, but that’s a workaround in the meantime#2018-03-1613:06dnolen@alexmiller the behavior of -R seems a big strange to me, shouldn’t it compose any aliases + your local deps.edn deps?#2018-03-1613:37alexmiller-R is solely about selecting aliases to be used when resolving deps. The aliases it can select from are those produced by the combined deps.edn files in play, which typically includes your install, user, local deps.edn (and -Sdeps passed on command line)#2018-03-1613:37alexmilleraliases are not pulled from transitive deps#2018-03-1613:39alexmillerI’m not sure that would even be possible based on the order in which things are resolved#2018-03-1705:59seancorfieldJust FYI, I put my ~/.clojure/deps.edn file on GitHub https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#2018-03-1708:06dominicmI've been using the :user/ namespace for aliases on my home deps file, with the idea that it will prevent collisions.#2018-03-1814:40hypirionSo I have a (pruned) dependency tree that looks like this
.
├── a 1
│ └── b 1
│ └── x 2
│ └── y 1
└─── c 1
└── x 1
└── z 1
and my result when doing clj -Stree is the following:
org.clojure/clojure 1.9.0
org.clojure/core.specs.alpha 0.1.24
org.clojure/spec.alpha 0.1.143
c path-to-c 1
a path-to-a 1
b path-to-b 1
x path-to-x 2
y path-to-y 1
z path-to-z 1
I'm guessing this is a bug? I would love to send a reproducible example, but it's a bit hard without polluting my GitHub profile. Maybe adding an option to include local git repositories for :git/url would be an idea?#2018-03-1815:10gfrederickswould a gist pollute your github profile?#2018-03-1815:13alexmillersome things that would be helpful:
- version info from clj -Sverbose (I’m assuming you’re on latest, but if not, then there were significant changes to resolution in 1.9.0.309)
- output from adding -Sdeps '{:aliases {:v {:verbose true}}}' -R:v#2018-03-1815:14alexmillerwith that I probably don’t need the reproducible deps.edn#2018-03-1815:16alexmillerthis is the trickiest kind of version resolution issue - when you have different versions of a dep and the lib sets (not just the versions) are changing across them.#2018-03-1815:17alexmillerI’m assuming that what you expect here is to have a1, b1, x2, y1, c1 as the output (and no z1)#2018-03-1815:31hypirionversion = 1.9.0.358
install_dir = /usr/local/lib/clojure
config_dir = /home/jeannikl/.clojure
config_paths = /usr/local/lib/clojure/deps.edn /home/jeannikl/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/674722798.cp
Nothing in /usr/local/lib/clojure/deps.edn or /home/jeannikl/.clojure/deps.edn
@gfredericks: That's a good idea, forgot about gists#2018-03-1815:32hypirionAlso yes, I'd expect z1 to not be present#2018-03-1815:45alexmiller@hypirion I reproduced the simple example in a test case and can’t repro https://github.com/clojure/tools.deps.alpha/blob/master/src/test/clojure/clojure/tools/deps/test_alpha.clj#L100-L113#2018-03-1815:46alexmillerwhich leads me to believe its most likely a path comparison issue with git deps#2018-03-1815:46alexmillerthe verbose output should be sufficient for me to track that down#2018-03-1815:49hypirionHuh, it seems like it depends on the order of the elements in the map.#2018-03-1815:49hypirionAlmost done with the gists now, one sec#2018-03-1815:54hypirionhttps://gist.github.com/hypirion/b955e63ebb394c216c61fe1b1b6a48fd
If I flip the dependencies around in that repo, it doesn't include z.#2018-03-1815:59hypirionI'm still a bit unsure about how the algorithm works, but I think it should be possible to walk the dependency tree at the end. If you can't reach a dependency by walking the tree, then don't include it as a dependency you need.#2018-03-1816:03alexmillerI have to run but will look at it later#2018-03-1816:04alexmillerThat’s being done but it relies on comparing deps up the path#2018-03-1816:06alexmillerIf the same dep has two different representations then that path matching will fail. That shouldn’t happen, but that’s my suspicion#2018-03-1816:09hypirion@alexmiller: I flipped the order of the dependencies in your test case and mvn test fails on my machine with that change. https://github.com/hypirion/tools.deps.alpha/commit/5459773c7236e37d0306d1561d5dc5a42243ca70#2018-03-1816:16alexmillerInteresting, well I guess I was just lucky :)#2018-03-1816:22hypirionheh, yeah. Dependency algorithms are hard to get right#2018-03-2016:04ghadiDoes anyone use cljfmt via clj?#2018-03-2017:00delaguardoI can use it like that
clj -A:cljfmt -e "(require '[leiningen.cljfmt])" -e "(leiningen.cljfmt/cljfmt {:source-paths [\"src\"] :test-paths [\"test\"]} \"check\")"
with an alias inside on deps.edn
:cljfmt {:extra-deps {leiningen {:mvn/version "2.8.1"}
lein-cljfmt {:mvn/version "0.5.7"}}}
#2018-03-2017:21ghadiHero, thanks#2018-03-2120:45kidpolloTo be able to use :git/url sources the target repo has to also have a deps.edn file right?#2018-03-2120:46alexmillerit needs to either not have deps or declare them with an understood manifest format#2018-03-2120:47alexmillercurrently tools.deps understands deps.edn and pom.xml#2018-03-2120:47alexmilleror you can force deps.edn semantics with no deps by using :deps/manifest :deps#2018-03-2120:47kidpolloI see. What about local mvn sources. I am having a hard time getting Artifactory sources working with deps :S#2018-03-2120:48alexmillerwhat does “local mvn sources” mean?#2018-03-2120:48danielcompton@kidpollo note that you won't be able to auth to Artifactory until https://dev.clojure.org/jira/browse/TDEPS-9 is fixed#2018-03-2120:48kidpolloI am looking for a workaround for the “no username password sources supported”#2018-03-2120:50kidpollothanks for pointing that out @danielcompton#2018-03-2120:50alexmilleryou could mirror to an s3 repo accessed via aws creds#2018-03-2120:50alexmillerthat’s supported#2018-03-2120:50kidpollook thanks!#2018-03-2121:20danielcomptonI'm eagerly awaiting TDEPS-9 to be fixed, a few https://deps.co customers need it before they can start using it#2018-03-2121:21danielcompton@alexmiller is there anything I can do to help there, or is it just waiting for review?#2018-03-2121:21alexmillerit’s just down my queue a bit. We need it for Datomic too.#2018-03-2121:22alexmillerI’m on a consulting project these days so only getting a few hours of Clojure time per week atm. My priority right now is the Windows installer which is getting close.#2018-03-2121:22alexmillerAny testing or feedback on TDEPS-9 would be helpful.#2018-03-2121:28danielcomptonk, will take a look and see if I can verify that it works#2018-03-2121:48dominicm@alexmiller which patch is it you want testing/feedback on? Or both?#2018-03-2121:52alexmillerDon’t know, I haven’t looked at it at all#2018-03-2121:52alexmillermy only constraint is - no passwords in deps.edn files :)#2018-03-2121:54dominicmthat seems like it discounts patch a.#2018-03-2121:54alexmillerwell there you go#2018-03-2122:06dominicm@danielcompton I must say, I don't know if my patch works under java 9. I know reflection changed somehow in java 9, but I haven't taken the time to understand that change.#2018-03-2122:07danielcomptonI'll give that a go too 🙈#2018-03-2122:07dominicmYou've made yourself the expert on Java 9 by blogging about it one time 😛#2018-03-2122:13alexmillerBased on what I know, it probably does or can be made to in the context of clj#2018-03-2122:15alexmillerThe best way to test stuff like this is to hack your active clojure script to prepend your tools.deps.alpha/src/main/clojure dir to your lib path#2018-03-2122:16alexmillerThen you pick up all local changes immediately and can interactively test clj#2018-03-2216:03ghadigit-related
- https authentication is not supported yet by clj -- I wrote a patch that is waiting for eyes upstream
- ssh authentication via Java is a usability trashfire. Works easily for some, doesn't work for others. I started rewriting tools.gitlibs (the underlying library) to shell out to git, but I'm not sure it is viable because the solution must work across Windows to be accepted upstream.
- must ensure ssh-keyscan -H <host> >> ~/.ssh/known_hosts because you cannot accept unseen ssh hostkeys (clj is non-interactive)
- esoteric .ssh/config options can break cloning because the underlying java ssh libraries reads the config file poorly
- ed25519 keys supported via ssh-agent, but not via ~/.ssh/config entries
i wrote that internally, thought it would be a useful summary of some usability issues here#2018-03-2216:07alexmillerIf you want to make a troubleshooting page on wiki or site, go for it#2018-03-2216:16ghadiwiki page is a great idea#2018-03-2216:19dominicmReport from a training session we did with clj using the linux installer. The jars were created as root read-only! That makes the scripts broken for most users to our knowledge.#2018-03-2218:38ghadiwhat did you expect to happen?#2018-03-2219:14dominicmThe jars would be readable by everyone.#2018-03-2219:15ghadioh they're only readable by root?#2018-03-2219:15ghadithat's not what I see on my machine#2018-03-2219:17dominicmI think the script changed recently#2018-03-2216:21dominicmLooking at the script, I think it could be different to make these problems go away. Maybe I'll take a stab at tackling them.#2018-03-2216:21dominicm@alexmiller What is your process for testing the linux installer?#2018-03-2315:28borkdudeI just tested the linux installer in a docker container openjdk:10#2018-03-2315:31borkdudeThat worked. In openjdk:9 I got: Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty#2018-03-2315:32borkdude{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "60120f0f1420055051035103555406585006"}, :content ("[email protected]")}
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "ee9c81819aaedddc888ad9d8d788dcd6ddda"}, :content ("[email protected]")}#2018-03-2315:33alexmiller@dominicm sorry for non-response above - I saw it but got distracted before answering and then the notifier was gone so forgot about it. :) I don’t test the linux installer very well (just in a vm) and would appreciate any help in making it better.#2018-03-2316:15dominicmOkay, I'll have a think.#2018-03-2315:35alexmiller@borkdude if you’re trying to compare times, make sure that you’re comparing cached classpath in both cases by running more than once#2018-03-2315:35alexmillerthey mentioned that jvm startup times should be expected to be better in java 10 than java 9#2018-03-2315:35timgilbertPossibly the SSL error is this? https://github.com/docker-library/openjdk/issues/145#2018-03-2315:35borkdudeI’m comparing 8 and 9 here with just vanilla clojure after it downloaded the bootstrap dependencies#2018-03-2315:36borkdudeit seems to be a little bit better#2018-03-2315:36borkdudethis is openjdk, no idea if there is any difference with the Oracle JDK#2018-03-2315:36borkdudenot a JVM expert#2018-03-2315:37borkdudeit’s very easy to spin up a Docker: docker run -it openjdk:10 /bin/bash and then do the clj linux installer#2018-03-2316:06tianshuhow to do test with tools.deps?#2018-03-2316:07alexmillerWhat are you trying to test?#2018-03-2316:08tianshuI'm trying to use tools.deps instead of lein. I just want to run test like lein test.#2018-03-2316:09alexmillerCheck out the test-runner project #2018-03-2316:11tianshuthis is exactly what I'm looking for, thanks!#2018-03-2316:11alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-03-2316:11alexmillerOther useful tools#2018-03-2316:12tianshugreat#2018-03-2316:21tianshuwhen I use git/url, met an error
org.eclipse.jgit.api.errors.TransportException:
I just have get the :sha with git rev-parse HEAD.#2018-03-2316:21tianshuI'm confused what should I do?#2018-03-2316:22alexmiller@ghadi had a list of git connection issues in the back chat#2018-03-2316:23alexmillerIn this case looks like maybe you’re using the ssh url when the https would be better#2018-03-2316:23ghadihttps would work because that repo is public#2018-03-2316:24ghadiotherwise try ssh-keyscan -H >> ~/.ssh/known_hosts (I'm a bit surprised that https://github.com isn't in known_hosts -- perhaps you're containerized?)#2018-03-2316:24alexmillerI will update the test-runner read me to use https too#2018-03-2316:25alexmillerNo reason for an ssh url there#2018-03-2316:27tianshuI wonder where tools.deps put the git dependencies?#2018-03-2316:28tianshumaybe I can find it with clj -Spath#2018-03-2316:29tianshuokay, found it.#2018-03-2316:31tianshuthanks a lot.#2018-03-2317:00dominicmI opened an issue for that @alexmiller#2018-03-2317:58alexmillerwhere?#2018-03-2317:59alexmillernvm, I see it#2018-03-2323:17seancorfield@doglooksgood take a look at the deps.edn file in each of core.cache, core.memoize, and java.jdbc for examples of multiversion testing with clj and test-runner -- I recently converted all of those from Leiningen. #2018-03-2402:01tianshu@seancorfield It's very helpful. these project dose not have a resources folder, is there a resource-path in tools.deps or I should just use paths?#2018-03-2403:02seancorfield@doglooksgood the deps stuff is just about the class path. There's no distinction between the things that are on the class path. #2018-03-2403:04seancorfieldIf you use my boot-tools-deps then it has the idea of Boot resource path -- anything that would go in the uberjar -- and source path -- code that doesn't go in the uberjar. That puts deps paths into resource paths and extra-paths into source paths. #2018-03-2410:16dominicmWhy /usr/local > /usr/? Looking at https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard it suggests that /usr/local is wrong.#2018-03-2410:19dominicmAlso, the deps.edn files should be somewhere in /etc I think too? Reaching out here for some community feedback.#2018-03-2521:27danielcompton@alexmiller did you hear what Java versions people were running that had issues with http://repo.clojars.org?#2018-03-2600:03alexmillerNo#2018-03-2611:49mfikesNifty. The Planck homebrew build now depends on the Clojure homebrew install: https://github.com/Homebrew/homebrew-core/blob/9d13915487b6c817b0dd551b46243e8e95286774/Formula/planck.rb#L15#2018-03-2611:52mfikesAn interesting aside to the above: Even though Java isn't required to install or use the Planck binary, homebrew doesn't yet make a distinction, so you transitively need Java via Clojure to install the binary. https://github.com/mfikes/planck/issues/695#2018-03-2612:17roman01laHow to specify a dependency on a project within a subfolder in a git repo? Such as this https://github.com/bhauman/lein-figwheel/tree/master/figwheel-core#2018-03-2612:20alexmilleryou can use :deps/root attribute for the relative subdir in the repo#2018-03-2612:22alexmiller{:git/url ""
:sha "b3d1d16d4915948a36b341f5648e4c9763628ef3"
:deps/root "figwheel-core"}#2018-03-2612:26roman01lashould it work for Clojure 1.9? or do I need 1.10?#2018-03-2612:26alexmillerit declare the deps as needing only clojure 1.8#2018-03-2612:26alexmillerbut clojurescript 1.10.x#2018-03-2612:41roman01laI’m getting Error building classpath. nil when running clj with the following deps.edn
{:deps {figwheel-core {:deps/root "figwheel-core",
:git/url "",
:sha "b3d1d16d4915948a36b341f5648e4c9763628ef3"}}}
#2018-03-2612:42alexmillerI can’t reproduce that#2018-03-2612:42alexmilleris there a stack trace?#2018-03-2612:43alexmillerfrom clj -Sverbose, what version are you running?#2018-03-2612:43roman01la1.9.0.358#2018-03-2612:43alexmillerthat’s latest#2018-03-2612:44alexmillerdo you have anything in your ~/.clojure/deps.edn?#2018-03-2612:45roman01laThere was deps, but I’ve edited it to be just {} before running#2018-03-2612:45alexmillermaybe also use -Sforce to make sure you’re not getting something from cache#2018-03-2612:45roman01lastack trace https://gist.github.com/roman01la/8b7351d7697584a6bcf394adb5f722af#2018-03-2612:45roman01laok#2018-03-2612:46alexmilleroh interesting, that’s down in gitlibs. well, you could try cleaning your git cache in ~/.gitlibs#2018-03-2612:46alexmillereither selectively or by deleting the whole thing#2018-03-2612:48alexmillerthat looks like it’s getting a nil url, which is weird#2018-03-2612:55roman01laWorks now. Not sure how exactly, but after cleaning ~/.gitlibs it didn’t work first. Once I ran clj -Sforce it went fine and subsequent clj (without forcing) are ok now#2018-03-2612:55roman01laThanks!#2018-03-2613:07alexmillerI suspect something bad in your cpcache#2018-03-2615:47mfikes:+1: I managed to get clj to run in the Ubuntu PPA build for Planck. The main trickery was really ensuring it stays in the sandbox, which is a little tighter than Homebrew—you don't even have a home directory, for instance.#2018-03-2622:06richiardiandreaprobably late to the party but this is worth reading : https://hypirion.com/musings/a-resilient-git-dependency-algorithm#2018-03-2701:00ghadiwhat a great article.#2018-03-2703:12richiardiandreaYes#2018-03-2716:20dominicmI'm getting a weird issue. In Heroku, the clone step is failing for a https url with "authentication is required but no credentialsprovider has been registered". I can't think what the cause would be?#2018-03-2716:20dominicmMaybe it's something in ~/.ssh/config 😮 I don't know though.#2018-03-2716:32alexmillerthis is a git dep?#2018-03-2716:32alexmillerwith https url shouldn’t be using ssh at all#2018-03-2716:33alexmillerare you hitting a private repo via https? that’s not yet supported#2018-03-2716:47dominicmNo, it's a public one.#2018-03-2716:49dominicmOh yes, duh, of course ssh isn't engaged here at all.#2018-03-2716:52dominicm:extra-deps {
io.dominic/krei.alpha {:git/url ""
:sha "778ad4eb6130b7023204bf526b6f49d1429e4132"}}
Works locally, but not in heroku, it's definitely something contextual.#2018-03-2716:55hypirionhttps://github.com/SevereOveridfl0w/krei.alpha is a private repo it seems#2018-03-2716:56dominicmHaha. I see the problem now. There's a typo in my username. I'm so terrible. Sorry for the noise.#2018-03-2718:40jeff.terrellFor what it's worth, I learned recently that you can add custom ssh keys to Heroku, so that it can fetch deps from private git repos via SSH. (I had to do that because we were using a private npm package with github coordinates instead of npm registry coordinates.)
https://github.com/simon0191/custom-ssh-key-buildpack
(I realize you already solved your problem, @dominicm, it's just a useful capability to be aware of, so seemed worth sharing.)#2018-03-2718:41dominicmThat's very useful, thank you#2018-03-2721:51richiardiandreaSo spamming a bit more - in #clojurescript Alex gave me this brilliant/weird/maybe odd idea of converting https://github.com/boot-clj/boot/blob/30ba3ce69dcd8b2b3e00677a14db05c8ce49d59c/boot/base/src/main/java/boot/App.java#2018-03-2721:51richiardiandreaso that it can be called directly from clj#2018-03-2721:51richiardiandreaas in boot -m boot.main -param1 -param2#2018-03-2721:51richiardiandreaClojureScript does something similar at the moment and it is definitely a nice pattern#2018-03-2721:53dominicm@richiardiandrea @martinklepsch has already explored some of the issues here, https://clojureverse.org/t/boot-and-deps-edn/1266#2018-03-2721:53richiardiandreaoh ok cool didn't read that thanks#2018-03-2721:54richiardiandreaactually this is more in tune with my idea: https://clojureverse.org/t/boot-as-a-library/1432#2018-03-2721:56dominicmoh, that may be what I meant.#2018-03-2721:56dominicmclojureverse is down for me, so I was stabbing in the dark 😛#2018-03-2804:55semperosWhen specifying :git/url in my deps.edn, I appear to have to use http:// rather than https://. When using https://, the initial message indicates that it's cloning anonymously, but then I get error messages and stack traces showing that it's trying to use {:tag :mailto:gitgithub.comgitgithub.com, :attrs nil, :content nil}. Using http:// does not present this problem and the repo is cloned anonymously as expected.
## clj details
clj -Sdescribe
{:version "1.9.0.358"
:config-files ["/usr/local/Cellar/clojure/1.9.0.358/deps.edn" "/Users/dgregoire/.clojure/deps.edn" "deps.edn" ]
:install-dir "/usr/local/Cellar/clojure/1.9.0.358"
:config-dir "/Users/dgregoire/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
## deps.edn entry for depstar (that fails)
com.healthfinch/depstar {:git/url ""
:sha "4aa7b35189693feebc7d7e4a180b8af0326c9164"}
## Error that occurs when it tries to check out depstar
Cloning:
Error building classpath. #2018-03-2805:06alexmillerseems weird. can you use -Sforce too to make sure you’re not seeing something in cache?#2018-03-2805:17semperos-Sforce shows same behavior; deleted ~/.gitlibs and local .cpcache as well, no change#2018-03-2805:18seancorfield@semperos {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "66010f1226010f120e13044805090b"}, :content ("[email protected]")} does not indicate an https: URL -- that sounds like you've used an SSH URL instead.#2018-03-2805:20semperoscould be that it's late and I'm missing something obvious, but the :git/url in my deps.edn (pasted above) is ""#2018-03-2805:21seancorfieldThis worked for me {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "5e2d3b3f301e2d3b3f3073262e2d"}, :content ("[email protected]")}#2018-03-2805:22seancorfieldSo try a later SHA.#2018-03-2805:22semperosclj -Sdeps '{:deps {com.healthfinch/depstar {:git/url "" :sha "68f3a04f75c988fa032974e8c600c9086812e9ef"}}}'
Cloning:
Error building classpath. #2018-03-2805:23semperosnot limited to this repo, it's happening for others#2018-03-2805:23seancorfieldAre you on the latest clj / tools.deps ?#2018-03-2805:23semperosI pasted my info above, I believe I am; my $HOME/.clojure/deps.edn is blank and my /usr/local one is:
{
:paths ["src"]
:deps {
org.clojure/clojure {:mvn/version "1.9.0"}
}
:aliases {
:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.5.417"}}}
:test {:extra-paths ["test"]}
}
:mvn/repos {
"central" {:url ""}
"clojars" {:url ""}
}
}
#2018-03-2805:24seancorfield{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "750610141b350610141b580d0506"}, :content ("[email protected]")}#2018-03-2805:24semperosI deleted the cache in the local folder, I'll try clearing out the other cache dirs // no dice#2018-03-2805:26semperosclj -Sverbose
version = 1.9.0.358
install_dir = /usr/local/Cellar/clojure/1.9.0.358
config_dir = /Users/dgregoire/.clojure
config_paths = /usr/local/Cellar/clojure/1.9.0.358/deps.edn /Users/dgregoire/.clojure/deps.edn deps.edn
cache_dir = /Users/dgregoire/.clojure/.cpcache
cp_file = /Users/dgregoire/.clojure/.cpcache/2178403036.cp
Refreshing classpath
Clojure 1.9.0
user=>
#2018-03-2805:29seancorfieldI just tried it on OS X like you Tue Mar 27 22:28:33
(sean)-(jobs:0)-(~/clojure/semperos)
(! 504)-> clj -Sdeps '{:deps {com.healthfinch/depstar {:git/url "" :sha "68f3
04f75c988fa032974e8c600c9086812e9ef"}}}'
Cloning:
Checking out: at 68f3a04f75c988fa032974e8c600c9086812e9ef
Clojure 1.9.0
user=> ^D
Tue Mar 27 22:29:20
(sean)-(jobs:0)-(~/clojure/semperos)
(! 505)->#2018-03-2805:30seancorfield(sorry for the weird line breaks there -- I'm ssh'ing into my Mac from Ubuntu on my Windows 10 laptop!)#2018-03-2805:31semperosthanks for giving it a try; glad to know I'm the exception in this regard 🙂#2018-03-2805:34semperospicard-facepalm I apologize for the spam, I have an entry in my global ~/.gitconfig that was the culprit#2018-03-2805:35seancorfieldOh, interesting... what was it?#2018-03-2805:35semperos[url "
#2018-03-2805:37semperosif I do straight git clone things work as expected#2018-03-2805:37seancorfieldAh... that's good to know...#2018-03-2814:28ghadi@semperos JGit is too clever#2018-03-2814:28semperosmy thoughts too 🙂#2018-03-2818:10stephenmhopperHi, I have two questions. (1) Is there a way to make options required for tools.cli and (2) why doesn't the tools.cli repo have an issues tab? https://github.com/clojure/tools.cli#2018-03-2818:12ghadiClojure projects's issues are managed through JIRA, not Github -- see CONTRIBUTING.md#2018-03-2818:13ghadihttps://dev.clojure.org/jira/browse/TCLI#2018-03-2818:13ghadibug tracker ^#2018-03-2818:14stephenmhopperAh, I guess I didn't realize that JIRA was used for anything not in the core Clojure releases. Also, I think I might have answered my own question: https://dev.clojure.org/jira/browse/TCLI-12#2018-03-2818:35seancorfield@stephenmhopper Is the :missing approach sufficient for you? (and, BTW, tools.cli is not tools.deps / clj -- that seems to be a common confusion nowadays)#2018-03-2818:39stephenmhopper@seancorfield Yeah, :missing seems to be working for me. And yeah, I wasn't sure if this was the right channel or not. #clojure seemed too broad. Perhaps setting the topic for the channel would prevent confusion from folks in the future.#2018-03-2818:42seancorfield(The purpose mentions tools.dep api but that only shows when you have the Channel Details panel open)#2018-03-2818:42stephenmhopperCool. Thank you!#2018-03-2818:43seancorfieldI've also had folks open JIRA issues against tools.cli thinking it's the clj/`clojure` command line tooling 🙂#2018-03-2818:44seancorfieldI maintain tools.cli so I'm happy to field any questions you have about it.#2018-04-0121:50dominicmhttps://github.com/juxt/pack.alpha#auto-add-to-project I came up with an installer for pack this evening. It seems to work, it even preserves whitespace correctly. I'm not certain this is a good idea yet, but it certainly makes things easy.#2018-04-0122:28seancorfield@dominicm What is the final argument there? It looks like a SHA but doesn't match anything else? What am I missing?#2018-04-0205:28dominicmIt's the latest version sha. The sha of injector is newer than the code for it. They can probably match (I could potentially even automatically pick it up using horrible hacks)#2018-04-0202:22ehashmanNot quite a tools.deps question, but I'm not sure of a better place to ask---does anyone have any tips on debugging why an AOT-compiled app would be re-JIT compiling when run?#2018-04-0202:23ehashmanI have not looked inside the jar to check the timestamps, but I don't currently have a reason to believe something modified the source code timestamps to be different than the compiled artifacts#2018-04-0202:27alexmillerThere is a bug in the maven shade plugin that doesn’t preserve time stamps but not many people use that #2018-04-0203:00mikerod@ehashman https://groups.google.com/forum/m/#!topic/clojure/12BOjg1KsWA#2018-04-0203:01mikerodThat is a related discussion I had on this quite a while back#2018-04-0203:02mikerodWhen I was using maven shade plugin, my solution was to use maven exec plugin to open the shaded jar back up, bump all clj source time stamps back to something in the past and repack the jar. Did that in a script. Hacky#2018-04-0203:03mikerodYou could also do the workaround there about not including source clj files #2018-04-0203:05kidpolloare there any guides to using deps + cider?#2018-04-0203:05kidpolloI am having trouble getting it going with refactor-nrepl#2018-04-0203:50seancorfield@kidpollo Did you try asking in #cider ?#2018-04-0203:50seancorfield(I haven't tried it -- I only recently heard that CIDER supports deps.edn)#2018-04-0204:41kidpolloit does but cant get the refactoring tooling to work#2018-04-0204:53seancorfieldHmm, I'm not sure why deps vs lein/boot would affect the refactoring tool...?#2018-04-0205:41dominicmcider-jack-in, last I checked, hardcoded the deps list. It hadn't yet started using plugins properly.#2018-04-0220:26dominicmhttps://github.com/juxt/pack.alpha#auto-add-to-project now supports updates when an alias for pack exists, regardless of name.
I really like the approach that I'm taking here, I think it has legs. I could even see an alternative version of -Sresolve-tags based on this, which doesn't alter your indentation at all. That would be quite valuable, given how varied the indentation preferences are in the community.#2018-04-0518:08phreedhttps://github.com/RickMoynihan/lein-tools-deps is currently broken, I am trying to fix it.
The problem is in the call to https://github.com/clojure/tools.deps.alpha/blob/master/API.md#resolve-deps .
Expanding org.clojure/clojure {:mvn/version 1.9.0}
make-session nil /home/fred/.m2/repositoryjava.lang.NullPointerException: null
at clojure.tools.deps.alpha.util.maven$make_session.invokeStatic (maven.clj:106)
clojure.tools.deps.alpha.util.maven$make_session.invoke (maven.clj:102)
The system https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions/maven.clj#L74 variable is nil.
As you can see the system variable should be created by https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj#L84
When I run the relevant stuff from the repl (via clj) it all works fine.
https://github.com/phreed/lein-tools-deps/blob/master/scratch/plugin.clj
But from within the plugin I get the error mentioned above.
https://github.com/phreed/lein-tools-deps/blob/master/src/lein_tools_deps/plugin.clj#L45
Any insight?#2018-04-0518:18alexmillernote that lein-tools-deps is using a much older version of tools.deps.alpha than master#2018-04-0518:22alexmillerI’m not sure what the problem is that you’re seeing, maybe something to do with loading the right extension namespaces first (some of that is now done for you in current tda but did not used to be)#2018-04-0519:13phreedI am working with a fork of lein-tools-deps which is using 0.5.418-SNAPSHOT
https://github.com/phreed/lein-tools-deps/blob/master/project.clj
https://github.com/phreed/lein-tools-deps/blob/master/deps.edn
Here is the failing line...
https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj#L84
Here is the function that is failing
(:import
;; maven-resolver-api
[org.eclipse.aether RepositorySystem RepositorySystemSession]
...
(def the-locator
(delay
(doto (MavenRepositorySystemUtils/newServiceLocator)
(.addService RepositoryConnectorFactory BasicRepositoryConnectorFactory)
(.addService TransporterFactory FileTransporterFactory)
(.addService TransporterFactory HttpTransporterFactory)
(.addService TransporterFactory WagonTransporterFactory)
(.setService WagonProvider CustomProvider))))
(defn make-system
^RepositorySystem []
(.getService ^ServiceLocator @the-locator RepositorySystem))#2018-04-0519:16phreedApparently the-locator is nil.#2018-04-0519:23alexmillerWell that should never be :)#2018-04-0519:24alexmillerWhat’s up with the other deps - do those match what tda is using?#2018-04-0519:24alexmillerThose seem to be the old aether libs not maven-resolver#2018-04-0519:26phreedWhat is tda?#2018-04-0519:26phreed...and what other deps?#2018-04-0519:28phreedtools.deps.alpha#2018-04-0519:30phreedYou mean [org.eclipse.aether/aether-api "0.9.0.M2"] ?#2018-04-0519:30phreedI put those in to try to fix the issue, it did not change anything.#2018-04-0520:16alexmillerAs you can see locator is a delay instance, so it should never be nil, even if it failed to connect#2018-04-0520:29phreedRight, I moved things around so I could see what was going on better and now I get...
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No matching method: newServiceLocator, compiling:(lein_tools_deps/plugin.clj:52:3)
on
(def locator
(MavenRepositorySystemUtils/newServiceLocator
(.addService RepositoryConnectorFactory BasicRepositoryConnectorFactory)
(.addService TransporterFactory FileTransporterFactory)
(.addService TransporterFactory HttpTransporterFactory)
(.addService TransporterFactory WagonTransporterFactory)
(.setService WagonProvider CustomProvider)))#2018-04-0520:33phreedWhich makes no sense
https://maven.apache.org/ref/3.5.2/xref/index.html#2018-04-0520:34phreedWhich is clearly in here https://mvnrepository.com/artifact/org.apache.maven/maven-plugin-api/3.5.2#2018-04-0520:42phreedAnd, when I use clojure.reflect it shows
| :name | :return-type | :declaring-class | :parameter-types | :exception-types | :flags |
|-------------------+---------------------------------------------------+-----------------------------------------------------------------+------------------+------------------+--------------------|
| newServiceLocator | org.eclipse.aether.impl.DefaultServiceLocator | org.apache.maven.repository.internal.MavenRepositorySystemUtils | [] | [] | #{:public :static} |
| newSession | org.eclipse.aether.DefaultRepositorySystemSession | org.apache.maven.repository.internal.MavenRepositorySystemUtils | [] | [] | #{:public :static} |#2018-04-0520:44phreedI am an idiot. I removed the doto as well as the delay.#2018-04-0520:48richiardiandreahello folks, I was wondering if somebody else is working on updates to the linux installer by any chance (e.g., use install) ?#2018-04-0521:35alexmillerThere are some outstanding patches that I intend to apply for next #2018-04-0522:15dominicmI wrote a patch for install after we had issues on training#2018-04-0523:49richiardiandreacool thanks for sharing, meaning to have a look at that after we had problems too here#2018-04-0520:59phreedIt is actually getService that is returning the nil https://maven.apache.org/resolver/apidocs/org/eclipse/aether/spi/locator/ServiceLocator.html#getService(java.lang.Class)#2018-04-0521:40phreedI upgraded leiningen to 2.8.1 (from 2.7.x) and the problem resolved itself. Maybe a transitive dependency on a maven library. Thanks.#2018-04-0521:54alexmillerYikes #2018-04-0522:55seancorfield@phreed FYI, if you use triple back-ticks around code, it's much easier to read in Slack like this
#2018-04-0806:44seancorfieldFYI [seancorfield/boot-tools-deps "0.4.4"] just went to Clojars: it bumps the tools.deps.alpha dependency to 0.5.417 to address the slf4j logger warning.#2018-04-0816:15mfikesI have a plk script that composes clojure (for the purposes of classpath construction) with planck. This is working out fairly nicely, but there is a hack I needed to do in order to support :main-opts, by parsing the output of -Sverbose 😱
It makes me wonder if, akin to -Spath, a -Smain-opts would be generally useful.
(The fragile hack is at the bottom of https://gist.github.com/mfikes/bbf8a9a5b9ed6ecac15a3bb75d2dc8b4 if curious.)#2018-04-0816:16mfikes(Or alternatively, more info in -Sdescribe could lead to a solution.)#2018-04-0818:07alexmillerHappy to consider additions to -Sdescribe - file a ticket if so#2018-04-0819:32mfikesAt the very bottom of /usr/local/bin/clojure, does it make sense to have the script do
exec "$JAVA_CMD" ...
in order to have the shell script launching the Clojure process terminate (or be replaced). I'm asking because TBH I don't fully appreciate the consequences and wondering if anyone familiar with this aspect of Unix might know.#2018-04-0822:25tianshuis it possible to get the paths in program when using tools.deps. I try to write a figwheel.clj build script, but don't want hardcode the paths.#2018-04-0823:24alexmiller@doglooksgood could you explain more about what paths you mean?#2018-04-0823:25alexmiller@mfikes et al - I have deprecated the INST jira project, please use TDEPS for anything installer / clj related. Since I work on both of these in tandem, it’s just easier for me to manage and prioritize one bucket instead of two. And if you find a link to the INST one, let me know so I can update it.#2018-04-0823:25mfikesWill do!#2018-04-0823:26tianshu@alexmiller I want the :paths in deps.edn.#2018-04-0823:28alexmillergenerally speaking, I would say yes. but the answer is somewhat subtle.#2018-04-0823:32alexmillerthere are really three different levels at work here:
1) clj command line script (bash) - fully environment aware with knowledge of multiple sources for deps.edn and caching concerns. The script knows how to find all deps.edn sources and invoke make-classpath
2) make-classpath program (Clojure) - a small program that knows how to read and combine things it is told by #1 to use tools.deps to build a classpath
3) tools.deps resolve-deps / make-classpath - know how to do dependency resolution etc#2018-04-0823:33alexmillerto find the same :paths that clj will use, you need at least the top two levels (and possibly the third, if you want to consider the modifications from :extra-paths in aliases)#2018-04-0823:35alexmillerbut you might be able to take shorter paths to do things that are usually enough - it depends what other constraints/assumptions you can rely on and how close to clj you need to be.#2018-04-0823:37alexmillerfor example, if you know your local project.clj always has the paths in it and that’s all you want, then you could just literally read the file as edn and extract the :paths key.#2018-04-0823:37alexmillerand you don’t even need tools.deps to do that#2018-04-0823:38tianshuI my case, I write a .clj file which is a script file to launch figwheel build, and this require a source-path that contains .cljs files. I want to specify source-path with aliases in tools.deps#2018-04-0823:40alexmillerthere’s no way to self refer to the paths via a magic variable or anything like that#2018-04-0823:43tianshuonce the clj or clojure started, the paths will never change?#2018-04-0823:47tianshuif it is, provide apis or variables can make something easier, like finding the paths or deps. so some general purpose scripts can be write and reuse. I think this can help people use their own project to customize their workflow.#2018-04-0823:52alexmilleras described above, there are a lot of nuances to how clj and tools.deps works#2018-04-0823:53alexmillerthis tool is designed to launch programs prepared with certain dependencies on the classpath#2018-04-0823:54alexmillerperhaps a better way to ask the question is - can you write a program that can be launched by this tool that will output the desired information?#2018-04-0823:55alexmillerlike clj -e '(System/getProperty "java.class.path")'
#2018-04-0823:55tianshuyes, that is.😶 sorry for my bad question.#2018-04-0823:56alexmillernot a bad question, just trying to rephrase to alignment with the goal of the tool#2018-04-0823:57alexmilleris the output of the command above the answer you seek? or at least the input? that could easily be filtered to sub paths of the current directory#2018-04-0823:58tianshuyes, I think this is what I seek!#2018-04-0823:59tianshusuch a simple way:joy:#2018-04-0900:00tianshuthanks very much!#2018-04-0916:20drewverleeI think it would be really cool to write/blog/livecast someone translating a project.clj to a deps.edn. I mainly think this is cool because im having to do it right now and its very education, by which i mean, i have lots of things I dont yet understand 🙂#2018-04-0916:24dominicmOh, I did it by regex#2018-04-0916:25dominicm:%s/\[\(\S\+\) \(".*"\)\]/\1 {:mvn\/version \2} is how I did it in vim.#2018-04-0916:25dominicmThen I also ran :%s/\[\(\S\+\) \(".*"\)\ \(:exclusions \[.*\]\)]/\1 {:mvn\/version \2\r\3}#2018-04-0916:25drewverleeis it that mechanical of a process? Its possible this hinges on my understanding of what deps.edn does vs what lein project.cljs do#2018-04-0916:26dominicm@drewverlee deps.edn only specifies dependencies. So that's the only part that really converts over from project.clj.#2018-04-0916:26drewverleeI guess your right, the only thing i have to transfer is the deps#2018-04-0916:26drewverleeyep#2018-04-0916:27drewverlee@dominicm
out of curiosity, for those trying to use deps.edn, what are they using for
1) profiles
2) build stuff
3) deployment
I’m guessing boot can still handle these responsibilities#2018-04-0916:27dominicmThere are tools, like pack.alpha which add uberjars on top.#2018-04-0916:28dominicmhttps://github.com/juxt/edge has an example of doing most of this.#2018-04-0916:28dominicmEdge doesn't have a pack example yet, bridge does: https://github.com/robert-stuttaford/bridge/#2018-04-0916:28dominicm(although it's out of date, see the pack readme!)#2018-04-0916:34alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/Tools has some tools#2018-04-0916:36alexmilleraliases can serve some of the same purposes as profiles. pack.alpha and depstar are two build options.#2018-04-0916:37alexmillerif you have consumers use git deps, you can skip build and deploy altogether :) (I am somewhat joking - this is not feasible for a number of cases)#2018-04-0917:13dominicm@alexmiller I've been thinking about krei, there's an API for performing java complication in the jdk, so in theory if I search for an edn file describing how to build from source, it could be feasible.#2018-04-0917:14dominicmHaving git dependencies is a lot like working on a mono repo, which is the pain point krei was designed to solve.#2018-04-0917:15dominicmI think it could work, in theory. Haven't tried that hard yet though.#2018-04-0917:24alexmillerWhat’s krei?#2018-04-0917:32dominicm@alexmiller Krei is a in-repl build tool. It scans the classpath for krei-file.edn files, and then builds sources on the classpath from that.
Currently it supports cljs and sass, because that's what I need for the itch I'm scratching. But I think it could work for java too. Protobuf is probably important to target also.#2018-04-0917:33alexmillergotcha#2018-04-0917:36dominicmMaybe I should split into manifests and try to parse information from pom files? Dunno.#2018-04-0917:43alexmillerwell I can tell you that parsing info from pom files suuuuucks#2018-04-0917:45dominicmOnly using edn it is!#2018-04-0919:16seancorfield@drewverlee Take a look at this commit where I converted core.cache from using Leiningen to clj/`deps.edn`: https://github.com/clojure/core.cache/commit/1f76bdf7321bce7c5d8c1424943afeccff0fb6d5#2018-04-0919:17seancorfieldAnd here's a slightly more comprehensive example from java.jdbc being switched over https://github.com/clojure/java.jdbc/commit/e8040925beddab6048873625cfe791341082290d#2018-04-0919:19seancorfieldSo :profiles => various :aliases, "build" can either use tooling built on top of deps.edn (for example, I use boot-tools-deps with Boot on top of deps.edn and there's a Leiningen plugin, which is very much a work-in-progress), or it can use the JAR builders that are listed on that wiki page...#2018-04-1115:01bhaumanas folks migrate to tools-deps are people still using a "resources" directory to indicate static resources?#2018-04-1115:02alexmillerprobably so#2018-04-1115:06bhaumanI'm trying to come up with reasonable initial instructions for where people should place their index.html#2018-04-1115:07bhaumanso I'll keep rolling with resources/public/index.html#2018-04-1115:08bhaumanalso I was surprised that if you place "resources" in :paths that "resources" gets appended to the classpaths of all the :local/root's#2018-04-1115:29dominicmthat sounds like a bug.#2018-04-1115:36alexmillerthat is a bug that has been mentioned before. possibly no one logged it as I had forgotten about it.#2018-04-1115:37alexmillernope it’s logged https://dev.clojure.org/jira/browse/TDEPS-52#2018-04-1115:38alexmillerdefinitely will fix#2018-04-1116:03bhaumanoh cool, I was worried that it was on purpose#2018-04-1116:09alexmillerno, definitely not#2018-04-1222:45hagmonkI spent some time trying to make AOT for my app work with tools.deps, finally watched dtrace for a bit and figured out the magic :)#2018-04-1222:45hagmonkIn this case I'm also pulling in rebel-readline and clj-async-profiler:
clj -Sdeps "{:deps {com.bhauman/rebel-readline {:mvn/version \"0.1.2\"} com.clojure-goes-fast/clj-async-profiler {:mvn/version \"0.1.3-SNAPSHOT\"}} :paths [\"classes\" \"src\"]}" -m rebel-readline.main
#2018-04-1222:46hagmonkOh, and one must make the classes folder in advance too#2018-04-1223:09hagmonk@alexmiller is it expected that maven deps with scopes other than "compile" are filtered out? Looking at https://github.com/clojure/tools.deps.alpha/blob/582aee5c7ac43991fb3154a4596910a9d3a0c4f3/src/main/clojure/clojure/tools/deps/alpha/extensions/maven.clj#L81#2018-04-1223:10hagmonkI found some deps on libs pulled from our corporate artifactory instance were missing, seemingly because they have the scope "runtime" set#2018-04-1223:33alexmillerThat’s intentional but still thinking about whether to include runtime#2018-04-1300:12seancorfieldWhat does a runtime scope mean for Maven?#2018-04-1300:33alexmillerIt’s for running things :)#2018-04-1300:34alexmillerBut not needed to compile#2018-04-1303:21hagmonkHum, well, that's an interesting distinction#2018-04-1303:22hagmonkIs the classpath assembled for a deps.edn based project intended to get it only as far as compilation? I hit this problem through a transient dependency, and I actually had to use lein deps :tree to discover the problem, because tools.deps.alpha completely filtered out the existence of the transient dependency#2018-04-1303:39alexmilleryeah, there’s a ticket for this already#2018-04-1303:40alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-26#2018-04-1303:40alexmillerfeel free to add your vote#2018-04-1303:49hagmonkwill do sir, thanks#2018-04-1306:52seancorfield@richiardiandrea made me do it! clj -Sdeps '{:deps {seancorfield/clj-new {:git/url "" :sha "653441d86834160124a94065c4d92a4a9ad12554"}}}' -m clj-new.create app myapp/clj-test The beginnings of a clj-based new project template generator...#2018-04-1313:16johnSo, I can just do clj -m clj-new.create figwheel myapp and it'll bring in the leiningen template and build it?#2018-04-1313:34johnWhen I do clj -Sdeps '{:deps {seancorfield/clj-new {:git/url "" :sha "653441d86834160124a94065c4d92a4a9ad12554"}}}' -m clj-new.create figwheel figfig#2018-04-1313:35johnI get
Failed with: Failed to read artifact descriptor for figwheel:lein-template:jar:RELEASE
Exception in thread "main" clojure.lang.ExceptionInfo: Could not load artifact for template: figwheel
Tried coordinates:
[figwheel/boot-template "RELEASE"]
[figwheel/lein-template "RELEASE"] {}
at clojure.core$ex_info.invokeStatic(core.clj:4754)
at clojure.core$ex_info.invoke(core.clj:4754)
...
#2018-04-1314:50richiardiandreaAll in clj ;)#2018-04-1315:30seancorfieldAs noted, work in progress. Needs more work before it can do more than the built-in templates. But, yes, the plan is that will work. #2018-04-1315:30johnniiiiiice!#2018-04-1315:33seancorfieldI think that fails because I haven't set up the full clj environment for t.d.a. yet (in particular the repos).#2018-04-1320:18seancorfield@U050PJ2EU OK, clj -Sdeps '{:deps {seancorfield/clj-new {:git/url "" :sha "a87bdc0347fe1523e8d85283d94b06f5dd68b2cb"}}}' -m clj-new.create figwheel figfig This should work now!#2018-04-1320:19johnHahah!#2018-04-1320:20johntworks!#2018-04-1320:20johnthis is newsworthy#2018-04-1320:20johnwe'll have to whip up some clj/cljs.main templates 🙂#2018-04-1320:24seancorfieldI plan to add clj-template support soon, so we can have clj templates that produce deps.edn-based projects.#2018-04-1320:46richiardiandrealet me try macchiato#2018-04-1320:48richiardiandreaworks! awesome 😄#2018-04-1306:53seancorfield(it currently generates boot templates by default -- but it should be able to generate any Leiningen template too)#2018-04-1306:54dominicmLol, that's great.#2018-04-1306:54seancorfieldHe made me itch. I had to scratch it.#2018-04-1306:55dominicmI spent far too long last night writing an ANSI parser, I feel the sentiment quite hard right now :)#2018-04-1314:52alexmillersomeone dropped a docker command to spin a linux box for testing clj installs here a couple weeks ago - anyone have that handy?#2018-04-1314:54dominicmI wouldn't use that for testing#2018-04-1314:55dominicmDocker runs as root within the container, you miss a key aspect of testing that way#2018-04-1314:55alexmillerI wasn’t only going to only use it#2018-04-1315:50mkvlr> docker run -ti openjdk:8 /bin/bash#2018-04-1315:50hagmonk@dominicm that's true by default but it's trivially disabled#2018-04-1315:54mkvlrapt-get update
apt-get install -y rlwrap curl
export CLOJURE_VERSION=1.9.0.341 && curl -O && /bin/bash linux-install-${CLOJURE_VERSION}.sh#2018-04-1315:55hagmonkalthough for the purposes of installing clojure, it looks like the getting started guide recommends root anyway#2018-04-1315:55mkvlr@alexmiller these two things should do or do you want a built image?#2018-04-1315:57dominicm@hagmonk running it is the difficulty#2018-04-1315:58hagmonkrunning it?#2018-04-1317:04dominicmRunning the script as non-root is the important part, to ensure that OS permissions are correctly set#2018-04-1315:58hagmonkdocker run -ti openjdk:8 bash -c "$(curl ) && clojure"#2018-04-1316:08hagmonkNow with @bhauman power up mode: docker run -ti openjdk:8 bash -c "$(curl ) && clojure -Sdeps \"{:deps {com.bhauman/rebel-readline {:mvn/version \\\"0.1.2\\\"}}}\" -m rebel-readline.main"#2018-04-1316:09hagmonkI have a docker container internally that already has a .cpcache for fast rebel readline startup in a container :) Useful for when I need a totally clean slate#2018-04-1316:32alexmiller@dominicm curious what you think about https://dev.clojure.org/jira/browse/TDEPS-64#2018-04-1316:33alexmillerdo you think the prefix dir should be created if it doesn’t exist?#2018-04-1316:38alexmillerwith the TDEPS-65 patch for the linux install, it will be created if it doesn’t exist#2018-04-1316:38alexmillerI think I’m ok with that behavior#2018-04-1317:03dominicmSee no reason not to add a guard if we want to ensure that the prefix exists already. I suppose it's keeping with other tools to not create leading directories by default. I ignored that comment because I was fairly certain that the current behavior didn't follow that.#2018-04-1317:27hagmonkI'd actually argue, just from memory and experience, that directories should be created if it they don't already exist. I can't recall the last time I supplied any directory or prefix argument to a tool and then thought "shoot, I have to manually make that directory"#2018-04-1317:27hagmonkBut that's just my own experience#2018-04-1317:29alexmillerthis is mostly an issue for when you are supplying the prefix dir#2018-04-1317:29hagmonkright, I'd expect that if I supply the prefix dir, it will be created if it doesn't exist#2018-04-1317:29alexmilleryeah#2018-04-1317:30hagmonkI'm trying to think of a case where I've done something similar recently#2018-04-1317:31hagmonkI guess maybe the classic case is ./configure --prefix=/usr/local/whatever#2018-04-1317:31hagmonkwhen using autotools#2018-04-1317:32hagmonkI checked the nearest autotools generated configure script …#2018-04-1317:33hagmonkthey definitely mkdir -p, although they do it in the most gloriously overwrought tradition of GNU autotools:
# as_fn_mkdir_p
# -------------
# Create "$as_dir" as a directory, including parents if necessary.
as_fn_mkdir_p ()
{
case $as_dir in #(
-*) as_dir=./$as_dir;;
esac
test -d "$as_dir" || eval $as_mkdir_p || {
as_dirs=
while :; do
case $as_dir in #(
*\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'(
*) as_qdir=$as_dir;;
esac
as_dirs="'$as_qdir' $as_dirs"
as_dir=`$as_dirname -- "$as_dir" ||
$as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \
X"$as_dir" : 'X\(//\)[^/]' \| \
X"$as_dir" : 'X\(//\)$' \| \
X"$as_dir" : 'X\(/\)' \| . 2>/dev/null ||
$as_echo X"$as_dir" |
sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{
s//\1/
q
}
/^X\(\/\/\)[^/].*/{
s//\1/
q
}
/^X\(\/\/\)$/{
s//\1/
q
}
/^X\(\/\).*/{
s//\1/
q
}
s/.*/./; q'`
test -d "$as_dir" && break
done
test -z "$as_dirs" || eval "mkdir $as_dirs"
} || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir"
} # as_fn_mkdir_p
#2018-04-1317:33hagmonkeww sorry, I thought that would collapse#2018-04-1318:37seancorfieldQuestion about classloaders -- when I run code with clj -m ... should I be able to get at a modifiable classloader?#2018-04-1318:46seancorfieldIt seems that when you're in the REPL, you have a DynamicClassLoader from Clojure, but when running via -m you only have sun.misc.Launcher$AppClassLoader and sun.misc.Launcher$ExtClassLoader neither of which are modifiable...#2018-04-1318:57dominicmThe goal should be to act like a normal Linux program. Regardless of whether we expect those behaviours ourselves.#2018-04-1319:04seancorfieldWell, to dynamically load Leiningen templates at runtime, I need a modifiable classloader... for now I've made one (`clojure.lang.RT/makeClassLoader`) and stuffed it back into the current thread.#2018-04-1319:46seancorfieldAnyone happen to know what package org.sonatype.aether.collection.DependencyGraphTransformer is part of? When I try to load leiningen-core via t.d.a. this doesn't seem to get loaded...#2018-04-1319:56seancorfieldNM, figured it out.#2018-04-1320:23johnour diabolical plans are almost in place#2018-04-1320:23seancorfieldQualified project names do not work (they don't work in boot-new either).#2018-04-1320:23seancorfieldOther known issues https://github.com/seancorfield/clj-new/issues#2018-04-1320:24johnthis is super sweet though, thanks Sean!#2018-04-1320:25seancorfieldI'll work on it more over the weekend and make an official announcement on Monday I expect.#2018-04-1320:54richiardiandreaDon't have to say how cool this is, no more lein or boot for template generation is a boon for lumo projects#2018-04-1321:14alexmillermight want to add that to https://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-04-1321:15alexmillernew tools.deps.alpha 0.5.425 is out. new installers working their way through the meat grinder#2018-04-1321:16alexmillermostly just bug fix releases for both, I’ll do a ggroups announcement when brew is done#2018-04-1321:19seancorfieldI'll add it to the tools page after it's had some polish this weekend. There are a few issues (opened on GH!) that need addressing before it's ready for real world users! 🙂#2018-04-1321:42johns/lein new/clj -A:new/#2018-04-1322:00seancorfieldclj -A:new right? 🙂#2018-04-1322:00seancorfield(or does it support omitting the leading : now?)#2018-04-1322:02richiardiandreaI have put it directly in my :deps in ~/.clojure/deps.edn#2018-04-1322:43johnma bad#2018-04-1322:44johnso what's it look like? the -m ... line goes in extra-args right?#2018-04-1322:44johnah, main-opts#2018-04-1322:55johnYeah, this works for me:
...
:aliases
{:new {:main-opts ["-m" "clj-new.create"]
:extra-deps
{seancorfield/clj-new {:git/url ""
:sha "a87bdc0347fe1523e8d85283d94b06f5dd68b2cb"}}}
...
#2018-04-1617:01timgilbertAdding this magic entry to ~/.clojure/deps.edn, or a project deps.edn, seems like something that could be written as a useful tool, similar to npm install or yarn add or whatnot - typing all of this info is slightly too annoying right now for use in shell scripts / one-liners IMHO#2018-04-1617:04dominicmI have this in pack.alpha#2018-04-1617:04dominicmPack can inject itself for you.#2018-04-1617:04dominicmI'm thinking it might become a general injector eventually which you can then do clj -A:user/inject new to inject into deps.edn#2018-04-1322:57johnthen just do clj -A:new figwheel figthing#2018-04-1323:03alexmillerI intentionally wrote the code to be a little lax on that leading colon so it’s optional#2018-04-1323:26seancorfieldclj-new SHA "87d175c5e49618fffe8ed16b1bf86aa6f0fbc624" -- https://github.com/seancorfield/clj-new/blob/master/README.md -- a whole bunch of clean up, refactoring, and rewriting the default project templates.#2018-04-1323:38seancorfieldThe built in templates now generate projects with deps.edn files $ clj -A:new app myapp
$ cd myapp
$ clj -m myapp.core
Hello World!
$ clj -A:test:runner
...FIXME test failure...
#2018-04-1400:24seancorfieldOK, something we've all wanted ever since Leiningen did the .core thing... clj-new SHA "f6fcc24bfa5d77167ff826990cd2c9c65eed4fed" prohibits single-segment or unqualified project names!#2018-04-1400:25seancorfieldclj -A:new app foo/bar or clj -A:new app foo.bar#2018-04-1400:25seancorfieldAnd, with that, I'm out for the day...#2018-04-1400:45richiardiandreaThat's great thanks a lot Sean!#2018-04-1407:56seancorfieldclj-new SHA "025f678af79c460e23f762aeb33dbb4af16f7ec7" now supports templates that are :git/url + :sha or :local/root (+ template name) -- https://github.com/seancorfield/clj-new/blob/master/README.md#2018-04-1407:56seancorfield(rewriting the Boot generator stuff will probably be tomorrow/Sunday at this point)#2018-04-1413:23johnSo what would be the pros and cons of moving some projects off private mavens completely and just moving to private git shas?#2018-04-1413:24johnor private git urls#2018-04-1413:24johnseems like you're duplicating your deps anyway, if you've got your snapshot in github#2018-04-1413:25johnWhile also just copying them to some yaven...#2018-04-1413:28johnin other words, unless I'm providing to some maven consumer that can't use clj, why use maven at all now?#2018-04-1413:33johnhmm, I wonder if a lot of clojure libs somehow depend on things living on maven...#2018-04-1415:04richiardiandrea@john I think that clj deps are fairly independent from maven but I might be wrong, lumo is experimenting with npm packages for instance: https://andrearichiardi.com/blog/posts/lumo-npm-dependencies.html#2018-04-1415:08dominicm@john None I've run into depend on things being from maven.#2018-04-1415:10mfikes@john One small thing, at least for now is that ClojureScript's :aot-cache doesn't yet work on Git deps. But that could change https://dev.clojure.org/jira/browse/CLJS-2651#2018-04-1415:11john@richiardiandrea @dominicm that's what I was thinking#2018-04-1415:12john@mfikes hmm. Does that just mean that some cljs things won't be cached on first pull of the dep?#2018-04-1415:12johnBut they still cache after that, right?#2018-04-1415:13johnor, if someone is using :aot-cache true then it'll break with git deps, currently?#2018-04-1415:14dominicmIt won't break. They just won't be candidates for caching.#2018-04-1415:15mfikes@john It just means that Git deps behave like things did before :aot-cache existed: They do still cache locally on first build.#2018-04-1415:15johnrgr#2018-04-1415:17johnso if you have a project where you're waiting to migrate to clj tool, because of the ongoing work on private maven repos, you might be able to just skip all that and depend on the git dep directly, right?#2018-04-1415:18dominicmAs long as you control the private project you're interested in, yes.#2018-04-1415:18johnObviously I won't fully know until I try migrating a whole project over, but I'm just doing a sanity check first#2018-04-1415:18dominicmTrue for internal projects, false for datomic 🙂#2018-04-1415:19johnright#2018-04-1421:16seancorfieldThis was something that I noticed as I worked on clj-new -- I forked it from boot-new so at first it still had a build.boot, which was the only place where a version number was stored, and was the only place that had anything to do with building artifacts and deploying them (to Clojars). And I was suddenly "Oh, wait! I don't need a Clojars badge on this project, or version numbers, or artifact coordinates... 👀 "#2018-04-1421:19mfikesIn ClojureScript there was a similar issue where it normally produces a version number as part of its build process, which caused a little grief with trying to use ClojureScript as a "buildless" git dep. But that has been resolved (parts of the system have been changed to cope without a version number).#2018-04-1421:24seancorfield@mfikes Some Leiningen templates expect to be able to get at Leiningen's version number, which it assumes is in META-INF/maven/leiningen/leiningen/pom.properties -- which is why both boot-new and clj-new have that tree structure in src.#2018-04-1421:25dominicmWould be interesting to know "your" version (as defined in the deps.edn) somehow. I guess it could be set by clj using system properties when starting a jvm... 🤔#2018-04-1421:25mfikesIn ClojureScript's case the solution we went with was to derive a fake version number from the hashes of all of the code in the source tree#2018-04-1421:26dominicmThat gives me 2 use cases for contextual information now: tools.namespace could use it for reloading, and to grab a common piece of build information: version.#2018-04-1421:27dominicmI suppose it would need to be burned into the resulting UberJARs... but I can take of that 🙂#2018-04-1421:26seancorfieldAnd also the leiningen.release namespace. I tried to incorporate Leiningen 2.8.1 but it has a namespace with a def that slurps one of its own files when you load it(!), so I stuck with 2.7.1. Looks like it'll be "fixed" in 2.8.2.#2018-04-1421:29mfikesMy takeaway: Anything that needs to be "built" can be a hinderance for just using code directly as a dep.#2018-04-1421:30mfikes(Including something as simple as a version number.)#2018-04-1421:47hagmonkhey this is germane to what I'm working on#2018-04-1421:48hagmonkI started patching depstar to support building thin jars and producing a folder of jar artifacts#2018-04-1421:49hagmonkfor the purposes of building a kafka connector (which requires a thin jar + deps in a dir), and for separating deps into a separate docker file layer for speedier builds#2018-04-1421:50hagmonkquestion I'm facing at the moment: should things like version numbers or aliases that configure how to build the jar be in deps.edn, or kept exclusively as command line arguments?#2018-04-1421:53hagmonkI don't see anyone shoving extra stuff into deps.edn, so I'm somewhat wary about storing build metadata in there#2018-04-1422:11seancorfieldThere are pros and cons to both.#2018-04-1422:12seancorfieldIf you have a project that expects to be built a particular way, it makes sense to have the build stuff in the project (so an alias for using depstar etc sounds reasonable).#2018-04-1422:12seancorfieldIf you have a project that can be used "as-is" but could be built using any tooling available, don't put the build stuff in the project.#2018-04-1422:12seancorfieldThat would be my guidance.#2018-04-1422:14hagmonkthanks @seancorfield … right now I'm patching in clojure.tools.cli support to see how that feels in use#2018-04-1422:15seancorfieldI haven't switched that over to deps.edn yet. It's on my list.#2018-04-1422:15seancorfieldBut I also want to switch it to .cljc for ease of maintenance.#2018-04-1422:16seancorfield(and drop pre-1.7 support)#2018-04-1422:20hagmonkHah, I just realized depstar is @ghadi’s handiwork. That's why this code is so tight :)#2018-04-1423:28hagmonkOk, it's a bit unpolished, but if folks want to give the thin jar / dependency copying support I added, start here:
clojure -Sdeps '{:deps {depstar {:git/url "" :sha "9524dccdb37ee85b79d3a505447d1e5101f1a73d"}}}' -m hf.depstar.core -h
#2018-04-1423:34hagmonkThe TL;DR version is that -m hf.depstar.core jar works like lein jar, and -m hf.depstar.core deps works like lein-libdir#2018-04-1500:57alexmillerfrom my perspective re deps.edn: don’t put arbitrary other gunk in it.#2018-04-1500:58alexmillerI make no guarantees that arbitrary stuff will get properly merged across multiple deps files and/or transmitted through the code#2018-04-1501:04hagmonk@alexmiller thanks, that’s a compelling reason to keep things out of there :)#2018-04-1501:05alexmiller(although I think it probably will right now)#2018-04-1501:15richiardiandrea@dominicm congrats for the install patch, feels way better. It is also nicer to see the names of the contributors in the log. It was missing before but it goes a long way to "thank" contributors. FWIW, I like this idea.#2018-04-1502:00alexmillerI have done it a couple times in the past, I just forget sometimes.#2018-04-1501:15hagmonkOh cool, TDEPS-26 was fixed too#2018-04-1502:01alexmillerfor the record… https://groups.google.com/d/msg/clojure/6vEgPw5EW4U/9uamLErFBwAJ#2018-04-1502:01alexmillerNew releases of tools.deps.alpha and the Clojure tools (clj, clojure) are now available. This is mostly a bug fix release.
To update, use `brew upgrade clojure` on Mac, or the Getting Started installation instructions on Linux.
Windows installers are coming soon(ish)! I have an initial version working and it's about 80% done, so only 80% to go. :)
Updates:
- TDEPS-73 detect invalid deps.edn file and improve error
- TDEPS-61 use Clojars CDN repo
- TDEPS-26 include transitive runtime deps in addition to compile deps
- TDEPS-71 better error if Java not installed - primarily for Linux installer (patch thanks to Mike Fikes)
- TDEPS-65 specify permissions on installed files in Linux installer (patch thanks to Dominic Monroe)
#2018-04-1502:35seancorfieldI ended up not working on clj-new today, but may work on it tomorrow. I still plan to make an announcement about it early next week either way (we have a major infrastructure upgrade on Monday so I suspect it'll be Tuesday when I write up the clj-new stuff).#2018-04-1502:36seancorfieldI did bump the t.d.a. dep to the new release tho' 🙂#2018-04-1502:36seancorfield(locally)#2018-04-1519:47hagmonkso I'm trying to build an uberjar targeting a kafka connector … for the life of me I can't get kafka to load it, it barfs on Could not locate clojure/core__init.class or clojure/core.clj on classpath.#2018-04-1519:47hagmonkcore__init.class is definitely in the uberjar, however#2018-04-1519:48hagmonkbuilding this as an example with depstar, so I'm trying to figure out if the uberjar put together is somehow wrong#2018-04-1519:48hagmonk[2018-04-15 19:47:23,641] INFO Loading plugin from: /connectors/postgres-plugin.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:176)
[2018-04-15 19:47:23,646] DEBUG Loading plugin urls: [file:/connectors/postgres-plugin.jar] (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:183)
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.lang.Namespace.<init>(Namespace.java:34)
at clojure.lang.Namespace.findOrCreate(Namespace.java:176)
at clojure.lang.Var.internPrivate(Var.java:153)
at thals.pgconnector.SourceConnector.<clinit>(Unknown Source)
#2018-04-1519:49hagmonkhow's this jar managing to get as far as clojure.lang.Namespace but then claiming it's unable to find clojure/core__init.class …#2018-04-1519:54hagmonkoh crap, it's the classloader thing#2018-04-1520:20dominicmCan you link me to the documentation on how kafka loads classes? I'm the author of pack, and I'd like to read about what I'd need to do to support their preferred format.#2018-04-1520:30hagmonksure#2018-04-1520:32hagmonkbut, unfortunately, I don't think it's related to the jar format. I believe they start a new classloader and load all the jars into that, which isn't visible to clojure because use-context-classloader is set to true#2018-04-1520:33hagmonkI'm trying to figure out if there's a way to get around this without resorting to java#2018-04-1520:34hagmonkin terms of how they want stuff packed, it's documented here: https://docs.confluent.io/current/connect/devguide.html#packaging#2018-04-1520:35hagmonkthe tl;dr version is that they can take uberjars, or a directory full of jar dependencies (excluding kafka runtime goop), including obviously the jar containing the connector implementation#2018-04-1520:55dominicmThe zip version would work better, and fits with the
Pack philosophy.#2018-04-1702:23cflemingAm I correct in thinking that deps.edn provides no way to build an artifact?#2018-04-1702:24cflemingOr even to specify co-ordinates for one?#2018-04-1702:33richiardiandreaI think you understand correctly. Or put in a better way I understand the same.#2018-04-1702:53dpsuttonHave you seen ghadi's depstar?#2018-04-1702:53dpsuttonhttps://github.com/healthfinch/depstar#2018-04-1702:57cflemingI have, but that just creates a jar#2018-04-1703:05cflemingAlso, is Clojure itself an implicit dependency? The examples don’t add it like lein does.#2018-04-1703:08dpsuttonThat does an Uber jar so I think it gets the deps of the clj runner I think#2018-04-1703:09dpsuttonIt removes itself though#2018-04-1703:09dpsuttonDepstar itself I mean#2018-04-1703:09cflemingOh, I meant in general - if I run a REPL, the Clojure dep comes from the runner?#2018-04-1703:10dpsuttonAs far as I can tell yes. I've never seen deps.edn specify a clojure version#2018-04-1703:11cflemingInteresting. A lot of this is going to be really weird for Cursive support 😐#2018-04-1703:12dpsuttonIt's been rough in CIDER so far as well. It feels a bit like boot. It's not super obvious how to best approach it#2018-04-1703:12cflemingYeah, the loose approach to project identity is tricky.#2018-04-1703:12dpsuttonI think @dominicm has done some work towards that end but it's still a little brittle#2018-04-1703:15dpsuttonKinda like boot you need to know what aliases or tasks to invoke#2018-04-1703:15cflemingYeah, boot is tricky for Cursive too 🙂#2018-04-1703:17seancorfield@cfleming clj -Sdescribe lists the deps.edn files that are read in.#2018-04-1703:18seancorfieldYou can also get that from tools.deps.alpha as clojure-environment (I think).#2018-04-1703:18seancorfieldMy bad: clojure.tools.deps.alpha.reader/clojure-env#2018-04-1703:18cflemingThe difficulty is there’s no consistent identifier for a module#2018-04-1703:18seancorfield(it shells out to clj -Sdescribe tho')#2018-04-1703:19dpsuttonDo you inject your dependencies like cider or do you run another process? I know cursive is big on static analysis#2018-04-1703:19dpsuttonI think we're gonna have to merge edn files from emacs lisp#2018-04-1705:29dominicmNope. -Sdeps. I already have this working in viml, elisp will be a doddle#2018-04-1703:20cfleming@dpsutton I don’t inject any dependencies.#2018-04-1703:20cflemingMost of Cursive’s functionality is static in the editor.#2018-04-1703:23seancorfieldFor clj-new I ended up having to ask clj what deps.edn files it would load, reading them, resolving them, making a classpath, and loading those files... environment (clojure-env)
all-deps (read-deps (:config-files environment))
...
(-> (deps/resolve-deps all-deps resolve-args)
(deps/make-classpath (:paths all-deps) {})
(str/split (re-pattern java.io.File/pathSeparator))
(->> (run! pom/add-classpath))))
#2018-04-1703:24cflemingWhere are git deps cloned to?#2018-04-1703:25cflemingIs there a local .deps directory or something?#2018-04-1703:43cflemingAh, ~/.gitlibs#2018-04-1703:43cflemingIs it possible to use file:/// URLs with :git/url?#2018-04-1703:44seancorfieldYou can use :local/root#2018-04-1703:44cflemingI can’t get it to work.#2018-04-1703:44seancorfield(instead)#2018-04-1703:44cflemingRight, but then I can’t pin a version.#2018-04-1703:45cfleming(although I’m mostly trying to avoid having to publish to github just to play around 🙂 )#2018-04-1703:45seancorfieldSo :local/root deps just stay where they are on disk, with no version. :git/url is cached to ~/.gitlibs per :sha. :mvn/version is cached to ~/.m2 with a real version.#2018-04-1703:56alexmiller@cfleming re prior question, the default Clojure version is specified in the install-level deps.edn (which is merged with ~/.clojure/deps.edn and local deps.edn). You can specify it in local deps.edn too if you like.#2018-04-1703:57cflemingI’m thinking about the case when a Cursive user imports a deps project and maybe doesn’t even have the CLI installed.#2018-04-1703:58cflemingWhere is the install-level stuff put? I guess that’s OS/packager dependent?#2018-04-1703:59alexmilleryeah, but clj -Sdescribe will tell you that#2018-04-1703:59alexmilleryou can use the tools.gitlibs library to determine where the gitlibs are being cached https://github.com/clojure/tools.gitlibs/blob/master/src/main/clojure/clojure/tools/gitlibs.clj#L23#2018-04-1703:59cflemingThe problem is that IntelliJ, at least on OSX, doesn’t typically know what the users’s shell path is.#2018-04-1704:00cflemingWill any of this work at all if the user doesn’t have the clj CLI stuff installed?#2018-04-1704:00alexmillerno#2018-04-1704:01cflemingThat’s a bummer.#2018-04-1704:01alexmillerI mean, some of it can be used directly via the lib but the lib really works in tandem with the scripts#2018-04-1704:01cflemingHmm#2018-04-1704:02alexmillerthe scripts are the bridge from the user’s environment and config files into the lib#2018-04-1704:02cflemingI’ll have to see if I can detect the install, and consider what to do if it’s not there or I can’t find it.#2018-04-1704:02alexmillerthe scripts are the only layer that know about (for example) classpath caching#2018-04-1704:03alexmillerbut they use the lib to build the cache#2018-04-1704:03cflemingSo there’s no way to programmatically determine the classpath from Clojure/Java?#2018-04-1704:04cflemingWithout exec’ing, I mean?#2018-04-1704:04cflemingI guess yes, but with no caching.#2018-04-1704:04cflemingHmm#2018-04-1704:04alexmillerthere is no simple answer to that#2018-04-1704:04alexmillerthe lib can compute a classpath given a configuration#2018-04-1704:05alexmillerbut the configuration is inherently a product of the environment#2018-04-1704:05cflemingThe environment will be different when scripts are invoked by IntelliJ#2018-04-1704:06alexmillerto some degree, yes. and I think the question is how much you let the user know that :)#2018-04-1704:06alexmillerI think the user expectation will be that they get the same effect as running clj in the root of the project#2018-04-1704:07alexmillerI’ve tried to provide a lot of tools to replicate some or all of what’s in the script layer from inside the lib (Sean is using many of those in the boot tooling)#2018-04-1704:08cflemingright, but even running clj in the root of the project will be different when run from IntelliJ.#2018-04-1704:08cflemingWell, the environment will be different, I have no idea if the differences will be significant.#2018-04-1704:09alexmillerthe only things that are critical are current working directory and path#2018-04-1704:10alexmillerthere are potentially some environment variables that can affect things but none of them are likely to be in use#2018-04-1704:10cflemingpath will be the thing that’s different.#2018-04-1704:10alexmillercan you explicitly add an option to point to the bin dir for clojure?#2018-04-1704:10cflemingIn general OSX doesn’t give apps access to env vars, and doesn’t execute shell init scripts anyway.#2018-04-1704:11cflemingI think I’ll have to.#2018-04-1704:11alexmilleron os x, if you install via brew (recommended) clojure will be in /usr/local/bin#2018-04-1704:12alexmillerwell, on linux too if you follow the script#2018-04-1705:31dominicmMost distro packages won't#2018-04-1704:12alexmillerwindows is … coming#2018-04-1704:13alexmillerand the install level deps.edn will be at /usr/local/lib/clojure/deps.edn#2018-04-1704:14alexmillerthat last is linux, mac will vary per brew install#2018-04-1704:15alexmillermac will be at something like /usr/local/Cellar/clojure/1.9.0.375/deps.edn when installed via brew. maybe there’s a symlink that#2018-04-1704:15alexmiller’s version independent, not sure#2018-04-1704:18cflemingOk, thanks.#2018-04-1704:22alexmillerI have tried to do what I can to make things discoverable and computable. if there’s things I can do to help more, I’m open to suggestions#2018-04-1704:27seancorfieldWith boot-tools-deps I actually pull the install-level deps.edn from the Clojure brew-install GitHub repo, into the JAR of the library. Looking back, that was probably a bad idea 🙂#2018-04-1704:28seancorfieldI probably ought to change it to use clojure-env instead!#2018-04-1704:31alexmillerit’s a bad idea :)#2018-04-1711:42mfikes@cfleming If it helps, I've been using Cursive for deps.edn projects by first doing clj -Spom and then using the resulting pom.xml#2018-04-1711:45mfikesI also imagine Cursive might be able to use the output of clojure -Spath, as it gives you the classpath, including paths for any :local/roots or :git/url / :sha deps#2018-04-1717:03richiardiandreaSo...say I have a colleague who does not trust brew install and would like to use the linux script for installing clojure...is it possible to do that?#2018-04-1717:03richiardiandrea(answer is no, we tried and install -D does not unfortunately work on mac)#2018-04-1717:13dominicmThe install really only does 5 steps, should be easy enough to do them manually.#2018-04-1717:28richiardiandrea@dominicm yep he probably will do that#2018-04-1717:40alexmilleryou’ll notice the brew install just runs an install.sh in the tar#2018-04-1717:40alexmillerso you can manually do basically the same thing#2018-04-1717:42alexmillerwith some effort and messiness, install.sh for mac and linux-install.sh could become the same script. I have so far not been able to convince myself that gives me any great value though and there are some useful reasons for them to be different.#2018-04-1718:08richiardiandrea@alexmiller thanks for answering, was wondering about that as well, good to know#2018-04-1718:32richiardiandreaso the difference about install on Macosx is only with -d Create directories. Missing parent directories are created as required. No -D (capital).#2018-04-1718:46dominicmBSD vs GNU install 😔 if we want to work on BSD we should support both really.#2018-04-1718:53dominicmI've seen some applications probe by using --help and grep#2018-04-1719:12dominicmhttps://www.freebsd.org/cgi/man.cgi?query=install&apropos=0&sektion=0&manpath=FreeBSD+11.1-RELEASE+and+Ports&arch=default&format=html for reference#2018-04-1719:21richiardiandreaI think for mac is even a bit different from BSD, I did not see -D at all#2018-04-1720:13dominicmHum, weird, annoying, standards, yay.#2018-04-1802:39mfikesPlanck now joins the deps.edn fray with a new plk script which is literally a rip-off of clj#2018-04-1802:42alexmillersomething something flattery#2018-04-1900:04zentropeIf I have :main-opts ["-e" "(println 'hello)"] I get an EOF while reading error. Is there something I’m missing with the “-e” option?#2018-04-1900:08gfredericksdo you get that with (println :hello) as well?#2018-04-1900:08zentropeHm. If there’s any whitespace in the expression at all, the error happens?#2018-04-1900:09zentropeYeah, :hello causes the issue, too.#2018-04-1900:09zentropeHowever, :main-opts ["-e" "(println,'hello)"] succeeds (note the , for white space).#2018-04-1900:10gfredericksah ha#2018-04-1900:10gfredericksI would call that a bug but I Am Not A Clojure Lawyer#2018-04-1900:10zentropeIt’s certainly principle of greatest surprise. ;)#2018-04-1900:14zentropeI can’t find it in Jira, if it already exists. Not that I have any skill searching Jira.#2018-04-1900:27mfikes@zentrope https://dev.clojure.org/jira/browse/TDEPS-56#2018-04-1900:30zentropeYes, it seems major to me, too.#2018-04-1900:31mfikesSweet , whitespace hack though. Awesome.#2018-04-1900:33zentropeIf found that in a @seancorfield example for deps.edn somewhere. I wondered why he had that in there. ;)#2018-04-1900:34seancorfieldYeah, I spent a while bashing my head on the whitespace errors before thinking of , in a drunken moment of defiance, late at night! 🍻#2018-04-1900:35gfredericksI have also used , to avoid quoting issues at some point in my life#2018-04-2003:43mfikesThat , is curiously useful. Here is another use: https://github.com/cljsrn/cljsrn-org/blob/master/script/build#L1#2018-04-2003:47mfikesTry as I might, I can't find another way to embed deps EDN in a shebang without the Corfield Comma#2018-04-2004:40seancorfieldI'm honored -- I have punctuation named after me :rolling_on_the_floor_laughing:#2018-04-2014:23johnCorfield comma 😂#2018-04-2014:26john@mfikes this is exactly the simple static site generator I was looking for! thanks#2018-04-2014:27johnthat's really simple#2018-04-2014:39dominicm@mfikes just a caution, using the shebang that way won't work on Linux#2018-04-2014:40mfikes@dominicm Yep. I know. I know.#2018-04-2014:41mfikesIt is a nice pattern for when you want a script to specify its own deps, but alas.#2018-04-2014:50dominicmoh god#2018-04-2014:50dominicmI came up with something that works#2018-04-2014:52dominicm#!/bin/sh
"exec" "clj" "-Sdeps" "{:deps,{markdown-clj,{:mvn/version,\"0.9.85\"}}}" "$0" "
Run as
❯ ./foo.clj '# Hello, world'
I don't know whether to share this or burn it#2018-04-2014:53dominicmYou can kill the commas too, just tested#2018-04-2014:54mfikesNice, so the extra stuff there is just interpreted as strings by the compiler that otherwise do no harm 🙂#2018-04-2014:54dominicmBingo#2018-04-2014:55dominicmI had issues with your md->html, the docs said it was md-to-html? Dunno if in cljs it's different, but thought I would warn you#2018-04-2015:49timgilbertThat is a cool hack (also slightly horrifying)#2018-04-2015:50timgilbertI'm wondering if various bits of clojure syntax farther down in the file would break bash, but I guess it really does just read line-by-line?#2018-04-2015:53mfikesMy guess is that exec seals the deal, replacing the script with the clj process, thus making it so no further bash interpreting is required#2018-04-2016:05alexmillerthis still pales in comparison to boot’s hack I think - it embeds a jar file in the bash#2018-04-2016:06mfikes😱#2018-04-2016:06mfikesLike, the binary image?#2018-04-2016:07alexmilleryes#2018-04-2016:07alexmillerjust go edit your boot script :)#2018-04-2016:07mfikesHackers gonna hack#2018-04-2016:07dominicmfun fact, zip readers scan the file until they find the header.#2018-04-2016:07dominicmCapsule also has "really executable jars"#2018-04-2016:07alexmilleryep#2018-04-2016:19mfikesI like your solution @dominicm; it works on macOS as well. I’m thinking of putting it in the Planck online docs given it is so useful. https://github.com/planck-repl/planck/issues/720#issuecomment-383147716#2018-04-2016:19dominicm@mfikes Go ahead! I'm also honoured you took the time to figure out my gh username 🙂#2018-04-2016:20dominicmI tested it with a posix-only (ish) shell, so it should work everywhere#2018-04-2016:24mfikesI forgot about the fact that it needs to be well-formed Clojure code and used single quotes above. Hrm. It still worked.#2018-04-2016:27dominicmI guess that is well-formed clojure code, you are doing (quote "{:deps …") (quote "$0") 😂#2018-04-2016:28dominicmyou're not doing what it looks like you're doing, but it works#2018-04-2016:29mfikesYep. Wow#2018-04-2016:29mfikesI think I’ll use \" as it is a little less mystifying#2018-04-2016:34dominicmIt's also way more annoying 😉#2018-04-2016:34pesterhazySo much progress in one day! Corfield Comma... Monroe's Shebang#2018-04-2016:35pesterhazyI think I'd use clojure instead of clj. The latter is wrapped in rlwrap, which seems unnecessary for scripts#2018-04-2016:51pesterhazyTrying to code-golf: #!/usr/bin/env bash
{ "exec" "clojure" "-Sdeps" "{:deps {markdown-clj {:mvn/version \"0.9.85\"}}}" "$0" "#2018-04-2016:51pesterhazynot sure that's any better though#2018-04-2016:53dominicmIsn't it exactly the same? 😂 Although I like the hack#2018-04-2017:10pesterhazyYeah, the real insight is in the different interpretation of strings by sh and clojure#2018-04-2017:24mfikesOddly, it works with ClojureScript, which I would have expected to take issue with some strings appearing before an ns form
#!/usr/bin/env bash
"exec" "clojure" "-Sdeps" "{:deps {org.clojure/clojurescript {:mvn/version \"1.10.238\"} funcool/tubax {:mvn/version \"0.2.0\"}}}" "-m" "cljs.main" "-re" "node" "$0" "#2018-04-2018:45ghadiupgraded clj today, from 359 to 375, and I ran into the error Error building classpath. Destination path "my-private-lib" already exists and is not an empty directory
Seems like this popped up recently.#2018-04-2018:45alexmillercan you explain more about the context?#2018-04-2018:55alexmillerwas this on a linux install or using clj?#2018-04-2021:24ghadilinux install, using clj#2018-04-2101:25alexmillerUsing prefix dir?#2018-04-2314:34nicolaHi i have monorepo with submodules - looks like deps does not follow second level of relative paths? Should it?#2018-04-2314:38nicolacleo/cleo ../backend
fhir/fhir ./sansara/fhir
#2018-04-2314:38nicolafhir is not found in path#2018-04-2314:41alexmillerCan you explain your setup?#2018-04-2314:54nicolaui/
deps.end
backend/
deps.edn
libs/
lib1/deps.edn
lib2/deps.edn
# in ui
backend {:local/root "../backend"}
# in backend
lib1 {:local/root "./libs/lib1"}#2018-04-2314:54nicolaso lib1 is not visible from ui#2018-04-2314:55nicolaAnd -Spaths is wrong $ui-dir/./backend/libs/lib1/src
$ui-dir/../backend/src
#2018-04-2314:55nicolalooks like it just ignores relative paths#2018-04-2314:57alexmilleryou had “bacend” above - just checking that was a typo here#2018-04-2314:57alexmillerrelative paths should work#2018-04-2314:58alexmillerand “.libs/lib1” should be “libs/lib1" or “./libs/lib1”#2018-04-2314:59nicolaI’ve tried both 🙂 - will check one more time#2018-04-2315:01alexmillerclj -Stree might be easier to read too#2018-04-2315:02nicolaYes i read it#2018-04-2315:02alexmillerthere might be an issue with relative local deps of relative local deps?#2018-04-2315:02nicolaAnd wrong path is generated 😞#2018-04-2315:02nicolait lost ..#2018-04-2315:03nicolaso i’ve got ui/./libs/lib1#2018-04-2315:03nicolainstead of ui/../backend/./libs/lib1#2018-04-2315:05alexmillerI can imagine how that would not work in the code#2018-04-2315:10nicolaShould i switch to absolute paths or try to pull request 🙂?#2018-04-2315:10alexmillerwell you can definitely use absolute paths#2018-04-2315:10nicolado you have some reader macro to read env in deps.edn?#2018-04-2315:10alexmillerwe don’t take PRs for this project, but you can definitely file an issue at https://dev.clojure.org/jira/browse/TDEPS if you like#2018-04-2315:15nicolaYes, that’s my case - are you planning to support this situation?#2018-04-2315:20alexmilleroff the top of my head, I don’t see why this shouldn’t work#2018-04-2315:20alexmillershouldn’t be made to work that is#2018-04-2315:20alexmillerso yes#2018-04-2315:20alexmillerI think I know where the problem is#2018-04-2315:21alexmillerbut would be great to have a ticket#2018-04-2315:24nicolayep, will do - thank you. Done #TDEPS-74 + repo to reproduce https://github.com/niquola/local-deps-bug#2018-04-2316:29nicolaIs there any way to inject env variable into deps.edn?#2018-04-2316:57delaguardoclj -Sdeps '$(cat deps.edn | echo)' mb? But I never tried that before troll#2018-04-2317:04dominicmI mentioned this before, it's because the paths are not resolved relative to dependency manifest, but are relative to where the JVM started.
Is it now the case that you're interested in supporting this? 😯#2018-04-2317:14alexmillerSorry, I don’t remember that conversation #2018-04-2317:28dominicmif tdeps-74 is likely to be fixed, I'm happy. It suits my mono-repo intentions 🙂 ¯\(ツ)/¯#2018-04-2407:08delaguardohttps://store.docker.com/images/clojure?tab=description
Official tools-deps based clojure docker image is released. docker run -ti --rm clojure:tools-deps clojure#2018-04-2409:42pesterhazyVery interesting!#2018-04-2412:13alexmillerIn what way is it official?#2018-04-2412:48bronsahttps://docs.docker.com/docker-hub/official_repos/#2018-04-2412:48bronsaofficial by docker#2018-04-2412:50mfikesHrm. Seems odd Docker would create an official thing using clojure as its name. (I admittedly don't understand the bigger picture.)#2018-04-2412:50gfredericksI think it's existed for a while as a leiningen image?#2018-04-2413:08delaguardoyes, it is. And another for boot as well. And official means that this image is served from the official docker registry and curated by docker maintainers how suppose to take care about upgrading software inside of the image#2018-04-2620:39phreedI am wanting to create an uberjar and upload it to a local nexus repository. I am planning on using https://github.com/healthfinch/depstar or#2018-04-2712:45niclasnilssonQuick question: Transitive git deps in deps.edn? I.e. a/deps.edn points to b on github and b/deps.edn points to c on github. Doesn’t seem to work without including c in a’s deps.edn. Not implemented or am I doing it wrong?#2018-04-2712:59hypirionThat should work, I've used that myself a couple of times#2018-04-2713:08alexmillershould work#2018-04-2713:10alexmillerif you want more information, try clj -Sdeps "{:aliases {:verbose true}}" -R:verbose -Stree, I can probably interpret that a bit more to see what’s up#2018-04-2715:47rickmoynihannew snapshot of lein-tools-deps deployed btw… Should feature first stages of git & local/root support etc.
YMMV: https://github.com/RickMoynihan/lein-tools-deps#2018-04-2801:00mfikes^ If you'd like to use lein-tools-deps from Cursive, it is now usable, and I've documented some workarounds for some path issues here https://gist.github.com/mfikes/f803fef3013927c376063a3d72b69d60 (cc @rickmoynihan)#2018-04-2801:20cflemingThanks @mfikes, I’m hoping to add deps support to Cursive soon, that’s helpful.#2018-04-2801:20mfikesSweet!#2018-04-2801:22mfikes@cfleming Of interest to you will perhaps be the call (reader/clojure-env) and how it fails to find /usr/local/bin/clojure, if you end up going down a similar path https://github.com/RickMoynihan/lein-tools-deps/blob/master/src/lein_tools_deps/plugin.clj#L19#2018-04-2801:23cflemingThanks - I haven’t read about any of that yet, but I assumed that Cursive would have trouble with the CLI stuff since IntelliJ doesn’t see shell env vars like PATH.#2018-04-2801:25mfikesYeah, in the above, the workaround is to instead launch the IDE from a terminal via open -a "IntelliJ IDEA"#2018-04-2810:19rickmoynihanThanks @mfikes I might link to those docs from the README. I’m an Emacs/cider user myself, though am familiar with that problem because of how OSX’s GUI environment doesn’t fork from a shell with your sourced env, like it does under classical Unix systems.
The problems in resolving those file locations were actually the rationale behind letting users provide their own chain of files instead of [:system :home :project], so I’m glad you’ve found that useful; though it’d be good to find a better solution. It’s a shame that tools.deps has to shell out to get that env & I’m definitely open to other solutions; though it’s really more of a problem for tools.deps to solve (if it can).#2018-04-2813:36mfikesYeah, I feel like lein-tools-deps is at a point where early adopter community members can actually make use of it and provide feedback related to the various use cases that may arise out there.#2018-04-2817:23potetmfwiw I’ve gotten by thus far by generating a pom (via clj -Spom) — you don’t get auto-importing, but it’s a very clear execution model#2018-04-2817:25mfikesYeah, I think one new thing that lein-tools-deps gets you is Git deps (`clj -Spom` doesn't seem to support that)#2018-04-2817:26potetmah yeh, makes perfect sense#2018-04-3001:17hagmonkhey so can someone riddle me this … I'm trying to replicate https://github.com/stuartsierra/log.dev/blob/master/project.clj in tools.deps, but finding it quite irritating#2018-04-3001:19hagmonkthe problem is that lein takes the top level list of :exclusions in project.clj and applies those to all dependencies, and seemingly to all transitive dependencies#2018-04-3001:20hagmonkit doesn't appear that deps.edn has a facility to achieve the same thing, meaning that one has to manually review the dependency graph, and if any transient dependencies include an unwanted dependency, one must include that dependency and put a specific exclusion in the maven coordinate#2018-04-3001:34alexmillerThat’s true#2018-04-3001:37hagmonkI put a demo repo here: https://github.com/hagmonk/log.dev.deps#2018-04-3001:38hagmonkIs there a more "tools.deps-ish" way to solve this problem?#2018-04-3001:56alexmillerNo#2018-04-3002:00alexmillerThis is not a common issue so I don’t see it very high on the priority list#2018-04-3002:01hagmonkHmm, it's an issue I often run into when trying to "rationalize" the logging situation in a project that has lots of Java deps#2018-04-3002:01alexmillerOne thing that may exist at some point is the means to specify a lib removal. Given that, this could be done with override-deps or possibly a root dep exclusion#2018-04-3002:04hagmonkIt's kind of gross that the presence of certain deps on the classpath causes side effects for logging frameworks. Perhaps this is best handled by a small tool that rewrites deps.edn to incorporate the necessary inclusions / exclusions#2018-04-3013:14pesterhazy@hagmonk wow log.dev looks great, I never knew that existed#2018-04-3016:32hagmonk@pesterhazy yeah! since I end up importing a lot of Java deps I frequently start projects with this set of inclusions / exclusions just so I know that Java logging will be sane#2018-05-0314:35mfikeshttps://twitter.com/mfikes/status/992049711069704192#2018-05-0314:35mfikes^ demo of using some latest WIP for lein-tools-deps within Cursive#2018-05-0314:37alexmillerwhat’s that’s :system :home :project from?#2018-05-0314:37alexmillerbecause I would call those :install :user :project instead#2018-05-0314:38alexmilleris that lein-tools-deps?#2018-05-0314:38alexmillercc @rickmoynihan ^^#2018-05-0314:43dominicm@alexmiller is lein-tools-deps relying on implementation behaviour in order to label those things, based on positional information?#2018-05-0314:43alexmillerpresumably#2018-05-0314:44mfikesYes @alexmiller https://github.com/RickMoynihan/lein-tools-deps/blob/master/src/lein_tools_deps/plugin.clj#L19#2018-05-0315:09rickmoynihan@alexmiller: Yes, it’s lein tools deps.#2018-05-0315:10rickmoynihanYes, good idea those are better names I’ll create a ticket to rename them.#2018-05-0315:11rickmoynihan@dominicm: It does indeed. Do you know of a better way right now?#2018-05-0315:11dominicmNo, there's a ticket open on JIRA. I'm waiting on it#2018-05-0315:12rickmoynihando you have the ticket number? I’ll create an issue to update lein-tools-deps when tools deps improve it.#2018-05-0315:12dominicmhttps://dev.clojure.org/jira/browse/TDEPS-57#2018-05-0315:12rickmoynihanta#2018-05-0315:14dominicmIn my case, I want to allow users of pack to run pack from outside their main deps.edn if they like, which means replacing the project-deps.#2018-05-0315:16alexmillerwe’re still kicking around this stuff#2018-05-0315:17alexmillerfrom one perspective there are fixed things and from another tools.deps doesn’t ultimately care where they come from. Right now the scripts know about specific locations, but tools.deps doesn’t.#2018-05-0315:18alexmillerbut there are some places where it would be useful to know the context, for example when resolving deps.edn files in local transitive deps, you want to interpret that in the proper context (with install and user but without the initial dep project deps.edn)#2018-05-0315:18alexmillerand probably without -Sdeps#2018-05-0315:20rickmoynihan@alexmiller yeah I agree entirely.#2018-05-0315:21alexmillerI am taking the long hammock on the best resolution to this#2018-05-0315:21rickmoynihanhaha very wise 🙂#2018-05-0315:21alexmillerwe’ve got some datomic tooling that wraps around all this and has some of the same tensions#2018-05-0315:21alexmillerbuild tooling that is#2018-05-0315:23rickmoynihanI’ve been meaning to ask on macos with the brew installer, it looks like /usr/local/bin/clj etc are sym linked to the latest version; would it not be possible for /usr/local/etc/deps.edn to also symlink to the latest system deps.edn?
It looked link the linux installer puts system in a well known location.#2018-05-0315:23dominicmthat's a bit exciting, datomic + tools.deps seemed unlkkely to me as I expected java to be involved.#2018-05-0315:24alexmilleron macos, some of that is dependent on how you use brew I think. while it will usually be there, you can change your prefix dir for brew#2018-05-0315:24rickmoynihan@mfikes: sorry I haven’t got around to looking at your PRs yet btw… I’ll try and get around to it.#2018-05-0315:24dominicm@rickmoynihan don't depend on where the linux installer puts it, some distro packaging requirements put it elsewhere.#2018-05-0315:24alexmillerand similarly we now accept a prefix dir for linux#2018-05-0315:25alexmiller@dominicm this is related to how datomic cloud is built#2018-05-0315:25alexmillerwrt dependency fetching#2018-05-0315:26dominicmNixOS and GoboLinux certainly don't use /usr at all.#2018-05-0315:26alexmillereventually some of that whole system may percolate into something public and generally useful but not sure#2018-05-0315:26rickmoynihan@dominicm: Yeah I know… just thinking an acceptable approach for lein tools deps, might be to do some default checks (in a fallback chain) with no explicit config - but let users override.#2018-05-0315:27dominicm@rickmoynihan I would expect most linux users to not be aware of where their distro chooses to put these files.#2018-05-0315:27alexmillercertainly, the install-specific location is a pain#2018-05-0315:27rickmoynihanwould obviously prefer not to do that#2018-05-0315:28alexmillerone option would be to get rid of the install file and move it into the code#2018-05-0315:28rickmoynihanI’d expect most linux users would; or would be able to find out. They can always use clj -Sdescribe, which can be documented in an FAQ for lein-tools-deps#2018-05-0315:29alexmillerif the base is loaded as a resource, then the scripts no longer care#2018-05-0315:29rickmoynihanthat’s a great idea#2018-05-0315:30alexmillerlikewise, you could restrict ~/.clojure/deps.edn to aliases only#2018-05-0315:30rickmoynihanresolving user is a lot easier too so install is the main problem#2018-05-0315:30alexmillerwell, not only aliases, but not :deps/:paths#2018-05-0315:31rickmoynihan@alexmiller: sorry not sure why that’s an issue…#2018-05-0315:32alexmillerit goes to what you need to consider when reading transitive deps.edn, and also to what -Srepro means#2018-05-0315:32alexmillerthere are a bunch of these things that interrelate#2018-05-0315:33rickmoynihanok I see what you mean#2018-05-0315:43dominicmThat would solve some issues I have in pack with reproducibility, I don't look at the system deps.edn at all.#2018-05-0316:23seancorfieldboot-tools-deps includes a version of the system deps.edn that gets pulled down from GitHub and baked in at build time. While that means you can technically use boot-tools-deps without needing clj etc installed, it's also a very bad idea: if you have clj installed, it could have a different system deps file and you could get different behavior.#2018-05-0316:24dominicm@alexmiller I am a little confused by what you were saying about building datomic cloud, are you able to elaborate.#2018-05-0316:28alexmillerno#2018-05-0316:29alexmillerit’s all internal and I didn’t work on it#2018-05-0316:24seancorfieldGiven that I often tell folks boot-tools-deps is only intended to add Boot-like functionality to an existing clj project, I should start relying on clj -Sdescribe (via the API call in tools.deps.alpha).#2018-05-0316:26mfikesOne problem with the API call is that clojure needs to be in your PATH.#2018-05-0316:27mfikesSo, this causes an issue if you happen to run things from Cursive on macOS, which ends up with a degenerate path that doesn't include /usr/local/bin#2018-05-0316:28mfikesThis essentially hacks things so that it looks in there: https://github.com/RickMoynihan/lein-tools-deps/pull/31/files#2018-05-0316:30alexmillerI think given the changes I described above you may be able to avoid all of those#2018-05-0316:30alexmillerin that tools.deps.alpha would already include the necessary stuff#2018-05-0316:30mfikes(In case it is useful, here is the degenerate path that Cursive gets: /usr/bin:/bin:/usr/sbin:/sbin)#2018-05-0316:31mfikesYeah, TL;DR is stuff is baked in, no need to really run clojure, etc.#2018-05-0316:31alexmillerthe tradeoff is that the install deps.edn then becomes “magical” in that it’s hard to see what’s in it#2018-05-0316:32mfikesDefinitely calls for some hammock time#2018-05-0316:32dominicmI imagine in enterprisey environments you would want to have a system override of maven/repos#2018-05-0316:32alexmiller💤#2018-05-0316:34seancorfieldChanging the subject slightly... at work we have a mono-repo with multiple subprojects and we're starting to convert those over to deps.edn from Boot (with external dependencies in EDN files, using a different/simpler format). One of the problems we have to tackle is that, currently, we use a version.properties file to "pin" version numbers for libraries (each line is of the form: group/artifact=x.y.z to specify the version).#2018-05-0316:35seancorfieldSince clj requires that :override-deps be in an alias (yes?), I created a versions/deps.edn containing a :defaults alias with :override-deps for everything we need, but then it's a little clunky to run: clj -Sdeps "`cat ../versions/deps.edn`" -A:defaults:test:runner
#2018-05-0316:36seancorfieldIf clj allowed us to specify extra config files, that would be nicer. Thoughts?#2018-05-0316:40dominicmAlias only files would solve that issue too I guess, if they were searched for in parent directories.#2018-05-0317:14alexmillerso the :default-deps is really about this problem of dep mgmt in the large#2018-05-0317:14alexmillerbut it is not a fully formed (or working) idea#2018-05-0317:15alexmillerso I guess I’m saying - we see this problem and think it’s important.#2018-05-0317:16alexmillerbut there’s not a complete answer there yet.#2018-05-0317:24seancorfieldGood to know. Thanks @alexmiller!#2018-05-0317:25seancorfieldOur transition from Boot to clj / deps.edn is likely to be very slow/gradual because of stumbling blocks like this.#2018-05-0317:28seancorfieldOur monorepo has about 30 subprojects with all sorts of cross-dependencies. Right now we have a Boot task that uses tools.namespace to deduce dependencies based on namespace usage, so we don't have to explicitly manage those cross-subproject dependencies, but as we're restructuring code, I'd like to see that become more explicit and streamlined.#2018-05-0317:29seancorfield(we built the namespace-based dependency analyzer because we initially had some strict rules about how code in a few of those subprojects could use code in other subprojects, to maintain separation via APIs etc -- but we've changed our position on that now)#2018-05-0317:31dominicmI had been excited to see what you would invent for managing the deps.edn files using the automatic dependency analysis. I'm glad to hear someone else is going down the same path as us.#2018-05-0317:36seancorfieldYeah, we've talked about using the existing analyzer code to generate the cross-dependencies as deps.edn files but I think I want those human-editable with comments etc and I don't want to deal with any Clojure source reader complex enough to retain whitespace/comments 🙂#2018-05-0317:39dominicmIf you did want to go down the whitespace/comment retention, I had surprisingly good success writing the injector for pack with rewrite-clj: https://github.com/juxt/pack.alpha/blob/master/src/mach/pack/alpha/inject.clj 140 lines gets an entirely whitespace-safe modification of the deps.edn. It even does the funky vertical alignment of parameters if you do that in the same map at any point.#2018-05-0318:06seancorfieldYeah, I looked at that code -- and that convinced me I don't want to do that 🙂#2018-05-0321:26rickmoynihan@seancorfield / @dominicm can you describe those dependency analysis systems at all? What exactly are they doing, and what problems are they solving? Are you building multiple applications? If so is a limitation not that all applications in the same mono-repo need to use the same version of any given namespace?
If that’s the case what is the motivation for doing the analysis at all? i.e. why not just package everything into the same jar, and rely on the entry point determining the loaded namespaces? Is it because some of your libraries/apps are opensource and others aren’t?
Curious because we’re planning to move towards more of a mono-repo approach too.#2018-05-0321:32seancorfieldAs an example, we have subprojects for low-level systems-y stuff, data mapping (our wrappers for JDBC), a core domain subproject, a web utilities subproject, and so on. Then we have subprojects for each actual app (api, auth, login, billing, etc). So any given app subproject relies on a particular combination of other subprojects (most of them) but we also have background processes that don't need any of the web stuff etc. The tools.namespace analysis looks at the project you want to build, finds all the namespaces in it (across all subprojects) and the list of namespaces they all require, transitively (the basic graph analysis, treating everything as "one big app"), then it takes the project you want to build and starts from those namespaces to identify which subprojects contain namespaces that are (transitively) required, then the dependencies EDN file in each of those subprojects are read to generate the full list of external libraries needed.#2018-05-0321:33seancorfieldDoes that explain it well enough?#2018-05-0321:35seancorfieldThe two main goals were so that we didn't have to explicitly write down the subproject dependencies (they change automatically as code requires a different set of namespaces in other subprojects) and we wanted to enforce certain code path exclusions (e.g., no apps are supposed to depend on billing directly, they're only supposed to use the billing-sdk, billing apps shouldn't use the code domain library except via a specific "DMZ"-style namespace that defines the API).#2018-05-0321:37seancorfieldThat second goal is no longer important as we've substantially changed our deployment / infrastructure plans since mid-2017. And, without that, the benefit of auto-tracking dependencies is questionable -- I'm leaning toward explicit dependencies more now.#2018-05-0321:40rickmoynihanok so every app uses the same version of everything in the monorepo?#2018-05-0321:41rickmoynihanWhy not to just put all the src paths into one big classpath, uberjar that, and rely on the require-tree to just shake out what’s not used?
Is it because you also wanted the rules to be enforced?#2018-05-0321:42seancorfieldYes, the rules were important when we started down this path.#2018-05-0321:42seancorfieldWe can't quite make an uberjar of the whole thing because there are some subprojects that do rely on different versions of external libraries.#2018-05-0321:43seancorfieldBut we could uberjar most of it into one big JAR and start it with -m ws.whatever.namespace.#2018-05-0321:43rickmoynihansub libraries, or sub apps?#2018-05-0321:44seancorfieldOur core domain subproject has several -main entry points already for background processes and cron tasks.#2018-05-0321:46seancorfieldThe auto-tracking of dependencies makes it a lot easier to move code into new subprojects (keeping the ns the same) which we've done quite a bit as we've teased apart some of the legacy code from the newer code. But overall it wouldn't be much work to just manually update deps.edn in each subproject as/when we do that.#2018-05-0321:47rickmoynihanand each of those -main entry points has different external library versions?
Sorry trying to understand whether you consider multiple -main’s the same app, or is it what makes a different app that it has the same library versions / classpath, or something else?#2018-05-0321:48rickmoynihani.e we have some apps with multiple -main entry points… but they’re really different aspects/modes of the same app.
We also have different apps altogether#2018-05-0321:49seancorfieldFor the most part, each web app is its own separate subproject, since we deploy different apps on different servers. A lot of the background stuff runs a different server (well, two servers), so we want it separate from the web apps, but mostly all together for the server deployment.#2018-05-0321:50seancorfieldWe have some different dependencies in our web apps: one is heavy on websockets, one uses Bidi, the rest don't. We have two web apps that use netty/socket-io but have to use different versions because ... history ...#2018-05-0321:51seancorfieldWe also have a couple of subprojects that are only used from our legacy code base, so we want to keep that code physically separate so we can just delete it once we retire the legacy apps.#2018-05-0321:52rickmoynihanyeah, that’s an issue for us doing what I was suggesting… i.e. the transitive dependencies start to conflict and then you’re in a world of pain#2018-05-0321:53seancorfieldRight, we're in the situation where the dependencies would only conflict in the top-of-tree subprojects, not the lower-level stuff.#2018-05-0321:54seancorfield(and that's because we've historically "pinned" the versions of most of our external libraries)#2018-05-0321:46dominicmWe can't do as you propose due to different deployment environments meaning that certain code paths completely don't work.
It's also nice to be able to actually remove one of the two hundred dependencies every now and then, which is easier when they're limited to certain boundaries.
Our deployable files are massive now.#2018-05-0321:46seancorfieldOur JAR files are about 30-40MB each.#2018-05-0321:47dominicmThere's a lot of substituting in our projects. So running A might have it's iProtocol that's incompatible with B.#2018-05-0321:48dominicmOur files are 5x that#2018-05-0321:53rickmoynihan@dominicm: can you explain what you mean by deployment environments influencing the code paths working?#2018-05-0321:59rickmoynihanI’m hoping to manage our apps as independent projects in the same mono repo; with some shared support libraries alongside; where all apps will effectively share the same version of the supporting code… but apps may have different 3rd party deps.#2018-05-0322:04rickmoynihanThinking out loud; I’ve been wondering how this stuff fits with something like integrant… though it could well be a terrible idea#2018-05-0322:10rickmoynihanactually it is a terrible idea#2018-05-0405:30dominicmIntegrant with aero is basically how we solve some of this. We built something that looks like Integrant using component.#2018-05-0408:13rickmoynihanYeah, we use integrant too; in one project via duct (so weavejester can write all the middleware config layer for us) and in another older project we retro-fitted integrant with aero… which in many ways I prefer to duct; but there are pros and cons.
What I was really getting at though with the integrant thought was that you could possibly imagine combining integrant and deps.edn into a hybrid that would give you dependency resolution on components and configure/wire them together at the same time.
I find it interesting how tools like integrant/component essentially make your require tree composable and configurable, and wonder if there’s a way to use integrant for packaging too. i.e. to turn integrant components (modules) into individually resolvable artifacts that something like tools.deps could then fetch and resolve for you.
It’s currently a very half baked notion, and most likely a terrible idea though 🙂#2018-05-0405:33dominicmOur problem is pretty much that a namespace can blow up if deployed where it can't read certain environment variables and look at certain files. These files change per library, with the same path. I guess there's some convention for the path. This is particularly noticeable in localization.#2018-05-0407:23sveri@rickmoynihan In our team using gradle we define every dependency explicitly, no matter if its an in-team-project dependency or an external one. So naturally I would try the same with leiningen / tools-deps / boot.#2018-05-0408:17rickmoynihan@sveri: Not sure exactly what you mean by defining every dependency explicitly? Do you just mean that you produce mvn artifacts for each subproject?#2018-05-0409:15sveri@rickmoynihan gradle has a concept of project dependencies. In the root build file you can define your projects which then can depend on another.#2018-05-0414:51alexmillerthis is kind of a wip / sneak-peek but I’ve implemented a dynamic add-lib that you can use at the repl (that also will download and add transitive deps)#2018-05-0414:52alexmiller$ clj -Sdeps "{:deps {org.clojure/tools.deps.alpha {:git/url \"" :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}"
Cloning:
Checking out: at d492e97259c013ba401c5238842cd3445839d020
Clojure 1.9.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"})
Downloading: org/clojure/core.memoize/0.7.1/core.memoize-0.7.1.pom from
Downloading: org/clojure/core.cache/0.7.1/core.cache-0.7.1.pom from
Downloading: org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.pom from
Downloading: org/clojure/core.memoize/0.7.1/core.memoize-0.7.1.jar from
Downloading: org/clojure/core.cache/0.7.1/core.cache-0.7.1.jar from
Downloading: org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.jar from
true
user=> (require 'clojure.core.memoize)
nil
user=> (add-lib 'clj-time {:mvn/version "0.14.3"})
true
user=> (add-lib 'clj-time {:mvn/version "0.14.3"})
false#2018-05-0414:52alexmiller^^ you can use that command at the top to try it yourself off the branch#2018-05-0414:53alexmillereverything about this is subject to change as Rich and I are still discussing many aspects of naming and use#2018-05-0414:56alexmillerthere is also a new libmap namespace that tracks the libs currently in use (these can be loaded via a property that points to a file which is set by clj, or by a property set to edn data - hook for tools, or by just inspecting the classpath and trying to understand it)#2018-05-0414:57alexmilleradd-lib will add new libs in the context of the existing libs, so it won’t (for example) download and add libs of transitive deps that have already been loaded. when you add libs, those are added to the runtime lib map and considered in subsequent adds#2018-05-0414:59dominicmThis is very cool, and useful!#2018-05-0414:59alexmillerthe primary use here is to add a capability to the repl environment to dynamically add libs. this does not give you the isolation of something like pods, but it’s a small useful step into a more live repl experience.#2018-05-0415:00dominicm@alexmiller this uses java properties you say?#2018-05-0415:01alexmilleryes - clojure.libfile (will now be set by clj) and clojure.libmap#2018-05-0415:02alexmilleralthough the fallback to classpath introspection actually does a reasonable job even if that’s not set in my experience so far#2018-05-0415:03dominicmPods may conflict with that, as java properties aren't localized to the pod. I don't have an alternative suggestion, but just a data point that occurs to me.#2018-05-0415:04alexmillerif you’re using pods, you don’t need this#2018-05-0415:05alexmillerclasspath introspection:#2018-05-0415:05dominicmI'm thinking of inspection of clojure.libfile for other purposes, e.g. finding gitlibs to ignore during a (reset)#2018-05-0415:05dominicmThat's a bad example in the context of pods, but I think it conveys my intent.#2018-05-0415:06alexmillerthe clojure.libfile points to the libfile corresponding to the classpath used when starting the jvm. that has a fixed answer.#2018-05-0415:07alexmillerif you’re using pods, you’re in a bubble under that.#2018-05-0415:07dominicmlibmap then, rather than libfile?#2018-05-0415:07dominicmhttps://github.com/RutledgePaulV/clj-embed wouldn't be able to convey clojure.libmap to it's runtimes?#2018-05-0415:08alexmillerlibmap is still jvm-wide so like libfile#2018-05-0415:08alexmillerbut I think it’s interesting to consider how to allow classloaders to build their own env#2018-05-0415:08dominicmMaybe I don't understand what libmap is, what is it?#2018-05-0415:08alexmillerit’s just libfile in edn form#2018-05-0415:09alexmillerso a tool could start the jvm, build the classpath via means other than clj and pass the libmap via data rather than file#2018-05-0415:10dominicmGiven the definition of "jvm-wide", then java properties are always right. My thoughts are that it's really closer to being about the classloader though, what libmap generated the current classloader.#2018-05-0415:11alexmillerthe runtime libmap is basically stored in a var right now. maybe interesting to consider whether that should be dynamic to allow per-thread context or something per-classloader to handle pod/clj-embed type cases#2018-05-0415:12dominicmSorry, this probably sounds very pedantic. This is really exciting, I'm happy about the feature even as it currently is.#2018-05-0415:12dominicmIt's definitely notable that due to how Clojure works, embedding always carries a new copy of Clojure. So "clojure local" is perhaps sufficient?#2018-05-0415:33alexmillerno, I appreciate the good pokes on this. we’ve thought a fair bit about the add-lib stuff but less about the runtime management of the libmap and so I’m still trying to work out the ideal shape of that. and we have other deeper plans for the libmap stuff so I want to get it right.#2018-05-0415:33alexmillerit’s reasonably likely that part will actually float into Clojure itself#2018-05-0415:34dominicmWhich part, sorry?#2018-05-0415:35alexmillerthe runtime libmap management (not add-lib)#2018-05-0415:37dominicmSo clojure.core will be aware of the deps.edn format? 🤔#2018-05-0415:37alexmillerit’s just a map#2018-05-0415:37alexmillerso, yes it will be aware of maps :)#2018-05-0415:39alexmillerin terms of doing semantic things - that doesn’t have to be in core. core’s job is to ensure that it’s loaded and available in a way that can be replied upon#2018-05-0415:39alexmillerI wouldn’t rule out core being aware of it but I don’t see a need for that yet#2018-05-0415:40alexmillerwell, maybe. we’ll see where things go. :)#2018-05-0415:42alexmillermaybe I should push this stuff into a blog post#2018-05-0415:47dominicmI'm a little confused. Will it be a state storage, which libmap will sit in as a key?#2018-05-0415:48dominicmBlog posts are always exciting, even if marked as "these ideas are alpha"#2018-05-0415:48alexmillerI don’t think I understand the question#2018-05-0415:48alexmillerright now it’s a var holding an atom holding the libmap#2018-05-0415:49alexmilleryou can look at the code on the branch, it’s not that long#2018-05-0415:49alexmillerhttps://github.com/clojure/tools.deps.alpha/compare/add-lib#2018-05-0415:50dominicmWhat will Clojure know about managing the runtime libmap though? I'm really confused by what it means for that to be in Clojure, sorry.#2018-05-0416:29alexmillerit means that anything using Clojure could look at the libmap (not just things using tools.deps.alpha)#2018-05-0416:42dominicmI see! So there would be a var which was mutated or something?#2018-05-0416:34alexmillerhttp://insideclojure.org/2018/05/04/add-lib/#2018-05-0416:37seancorfieldThe channel name is #tools-deps not #tools.deps BTW @alexmiller (last line of the blog post)#2018-05-0416:37alexmillergosh dang it#2018-05-0416:38alexmillerfixed#2018-05-0416:56alexmillerit’s a var holding an atom, so would just swap! the atom#2018-05-0416:57alexmillerbut I think there are interesting potential uses that are merely read-only#2018-05-0417:09dominicmI understand now. I was interpreting your words too broadly I think. This would solve the problem of pods.
If I'm thinking right, this would complicate Clojure becoming aware of class loaders in the future. Admittedly, this is something I am a minority who longs for this. #2018-05-0417:25alexmillerClojure only works because of custom classloaders so I don’t think I agree at all with that#2018-05-0417:25alexmillerI also don’t think this would solve the isolation aspect of pods#2018-05-0419:43dominicmI mean that you can't require multiple versions of a namespace from different class loaders.
No? The pod could be controlled to update the atom on start couldn't it?#2018-05-0420:02alexmillerI think you’re misunderstanding what is happening here#2018-05-0420:02alexmillerbut I’m not sure that I understand what you think it’s doing enough to clarify#2018-05-0420:03alexmillerthe libmap is not affecting anything about how the Clojure runtime currently works#2018-05-0420:05alexmillerit is assisting in opening a path to additively add jars to the DynamicClassLoader in use (this actually has existed in Clojure for a very long time with clojure.core/add-classpath which is deprecated and I think has not worked for a very long time)#2018-05-0420:08alexmillerpods in Boot provide isolation (different Clojure runtimes) and already do allow you to load different versions of a Clojure namespace at the same time in different pods as far as I understand them#2018-05-0420:12dominicmMy use of from there was poor. You can't have multiple versions of a library loaded into a single Clojure. You have to load a whole new Clojure.#2018-05-0420:14alexmilleryes and nothing is changing about that#2018-05-0420:14alexmillerI’d say that problem is way down the interest list#2018-05-0420:16dominicmI know. I was thinking about it that were ever to be "fixed". Yeah, I'm one of the few it bothers because I have interest in a constrained deployment environment, where dependencies aren't really possible.#2018-05-0420:22alexmillerI totally get why and when isolation is useful, but Clojure was never designed with that intent and there seem to be more important problems to solve#2018-05-0421:24mfikeshttps://twitter.com/mfikes/status/992514389042454529#2018-05-0508:22slipsetAs asked on the twitters:
https://twitter.com/slipset/status/992662387600887808#2018-05-0508:25slipsetReason I’m asking is sort of at least twofold:
1) If it’s sort of the same, is there anything to be learned by the clj-refactor impl
2) If it’s sort of the same, can add-lib be used as a replacement for the clj-refactor impel so we get consistent behavior.
3) most likely just an off by one error#2018-05-0508:26dominicmI'm certain it's almost entirely the same. I expect that in a few months this implementation will find it's way into refactor-nrepl.#2018-05-0519:11hagmonk@alexmiller Riddle me this: how does one add a maven dep and the corresponding sources and javadoc jars to the classpath? (needed for cider javadoc to work)#2018-05-0519:12dominicm@hagmonk https://dev.clojure.org/jira/browse/TDEPS-12#2018-05-0519:13alexmillerNot supported #2018-05-0519:13alexmillerCurrently you can only have one artifact per library#2018-05-0519:14alexmillerI’m not opposed to supporting it, just trying to figure out how#2018-05-0519:15hagmonkGot it#2018-05-0519:25dominicm@alexmiller would treating it as a unique library be insufficient?#2018-05-0519:28hagmonkI'd propose clojure.tools.deps.alpha/merge-alias-rules be extended with some way to delegate merge strategy to the extension type#2018-05-0519:29hagmonkUh, maybe not, that is only used to merge the same map key between different deps.edn sources#2018-05-0519:31hagmonkmaybe clojure.tools.deps.alpha/resolve-deps instead - when merging :extra-deps into :deps#2018-05-0519:34hagmonkyou still end up spitting out a map though, perhaps the key of that map needs to change from being the dependency path, to an opaque composite key that represents "a unique artifact". The dep path itself could move into the value (during resolution, not in deps.edn itself)#2018-05-0519:36alexmillerThat’s only used for alias types, which are not extensible #2018-05-0519:37alexmillerThe merge-alias-rules that is#2018-05-0519:38alexmillerTreating it as a unique library isn’t possible as the group/artifact is used for resolution#2018-05-0519:38alexmillerThis needs more thought and I consider it a lower priority than other stuff on the plate#2018-05-0619:32hagmonkOk, another riddle for you guys#2018-05-0619:33hagmonkIf I include :mvn/repos in deps.edn for project A, because one of the coordinates has to come from a private maven server, then I depend on project A from project B via :local/root, clojure -Sdeps dies saying it can't find the artifact in the private repo#2018-05-0619:35hagmonkI keep running into these instances that are encouraging me to go back to producing a maven artifact, rather than using :local/root or :git/url#2018-05-0621:36alexmillerThis is a known bug, haven’t had a chance to fix it yet#2018-05-0622:20hagmonkI created a little tool to help me convert troublesome project.clj files into deps.edn. There are probably a dozen of these but I couldn't find one that actually worked for me :)#2018-05-0622:20hagmonkhttps://github.com/hagmonk/depify#2018-05-0623:07mfikes@hagmonk That's cool. I tried seeing what would happen if you made it run directly as a git dep, but the command line gets quite large https://github.com/mfikes/depify#usage#2018-05-0623:10hagmonk@mfikes yah, I thought about doing that too, and wondered if you could specify the git coordinate + -Sresolve on the command line and make it work ... haven't tried yet#2018-05-0623:10hagmonkgit coordinate as in the repo location + ref#2018-05-0623:24hagmonkah: -Sresolve-tags, and no that doesn't work, since it wants to mutate the deps.edn file ¯\(ツ)/¯#2018-05-0818:24richiardiandreaI have a question, with this deps.edn:
{:paths ["src"]
:aliases {:dev {:extra-paths ["test"]}
:self-host {:extra-deps
{andare {:mvn/version "0.9.0" :exclusions [org.clojure/clojure]}
mount {:mvn/version "0.1.13-SNAPSHOT"}}}}}
and command:
clojure -Srepro -A:dev:self-host -Stree
shouldn't be org.clojure completly excluded from the tree?#2018-05-0818:24richiardiandrea(`mount` does not include it)#2018-05-0818:25alexmiller-Srepro is kind of in an in between state right now - it’s actually including the install deps.edn, but not ~/.clojure/deps.edn.#2018-05-0818:25alexmillerand the install one includes clojure#2018-05-0818:25richiardiandreauhm...#2018-05-0818:25alexmillerI do not expect this to always be the case#2018-05-0818:25richiardiandreaok that answers it#2018-05-0818:25alexmillerbut the meaning and use of -Srepro is kind of a topic of internal debate atm#2018-05-0818:26richiardiandreaso at the moment is there a way to work in isolation? without any interference with the global env?#2018-05-0818:26alexmillerthere is no way to completely exclude the install-level clojure dep#2018-05-0818:26alexmiller(although clojure doesn’t run without clojure so I didn’t expect that to be an issue)#2018-05-0818:28alexmillerwhat are you trying to do?#2018-05-0818:29richiardiandreaI am working with self-host ClojureScript so I don't need Clojure for running, but having it on the classpath might cause issues#2018-05-0818:29richiardiandrealet me debug and see if that's the issue though#2018-05-0818:29richiardiandreabut in general you probably might want to consider the use case#2018-05-0818:52alexmillerclj runs clojure.main (always) which requires a class on the runtime. How does self hosted avoid that?#2018-05-0818:53alexmillerThat is, the entire purpose of clj is to run a class on the JVM #2018-05-0818:54dominicm@alexmiller I'm guess -Spath is being used only#2018-05-0820:36richiardiandreaYes I am using it only for building the classpath#2018-05-0820:36richiardiandreaAnd I am using clojure for it#2018-05-0820:42alexmillerwell, you could avoid this by invoking tools.deps directly too#2018-05-0820:43alexmillerthat is, as a library, rather than via the script#2018-05-0820:45alexmillerif you use -A:deps that will add tools.deps.alpha and its dependencies to your classpath. you could even invoke the clojure.tools.deps.alpha.script.make-classpath script directly#2018-05-0820:54alexmillerclj -A:deps -m clojure.tools.deps.alpha.script.make-classpath --config-files /path/to/whatever.edn --libs-file /path/to/out.lib --cp-file /path/to/out.cp --jvm-file /path/to/out.jvm --main-file /path/to/out.main -R :self-host#2018-05-0820:55alexmillersomething like that - I think that script requires all of those and that they are absolute paths#2018-05-0821:06richiardiandreaOk that is a possible solution thanks#2018-05-0821:23alexmillerif you want to write code instead, you could drop down to the tools.deps.alpha layer and write a few lines of code instead.#2018-05-0900:34richiardiandreayeah I would really like to explore tools.deps.alpha a bit more, expecially the resolver part#2018-05-0900:35richiardiandreaone day...#2018-05-0913:05dominicm@alexmiller Can you think of a case where clj on a deps.edn file wouldn't resolve a dependency that tools.deps.alpha would resolve against a deps.edn file?
In particular, https://github.com/juxt/pack.alpha/issues/7#issue-321346647 something is going off to another repo. pack only considers the deps.edn, and doesn't merge the system one. For some reason it works if I add maven central, but what does that have to do with glassfish?#2018-05-0913:05alexmillerthe install deps.edn includes the central and clojars maven repos#2018-05-0913:06alexmillerhttps://github.com/clojure/brew-install/blob/1.9.0/src/main/resources/deps.edn#2018-05-0913:07alexmillerother than that, shouldn’t be any different#2018-05-0913:09dominicm@alexmiller I guess another way of describing this would be:
- I run clj against a deps.edn file - so everything should end up in .m2
- I run tools.deps.alpha against the same deps.edn and it fails as more needing more resolving#2018-05-0913:09alexmillerdoesn’t make sense to me#2018-05-0913:10dominicmIf I add the :mvn/repos to deps.edn, somehow it allows it to fetch from glassfish repo#2018-05-0913:10dominicmah, they commented https://github.com/juxt/pack.alpha/issues/7#issuecomment-387732402#2018-05-0913:11dominicmI'm not really sure what is going on here, and why this either needs to run every time (uncacheable) or doesn't get picked up by clj#2018-05-0913:12alexmillerI guess because once it’s in central they want you to get it from there?#2018-05-0913:12alexmillerdunno, weird#2018-05-0913:12dominicmyeah, real weird.#2018-05-0913:13dominicmThis definitely highlights that I need a good way to do -Srepro without the local deps.edn file ever being picked up though 😉#2018-05-0913:13dominicmOr rather, I need a way to pick up only the system deps.edn#2018-05-1020:59rickmoynihan@alexmiller: I’m bike shedding a little but what would you recommend calling the key for lein-tools-deps config. Currently it’s called :tools/deps which feels a little wrong, should it be :lein.tools/deps or :lein.tools.deps, or just :lein-tools-deps?#2018-05-1021:02rickmoynihanThis is the key for using in a project.clj for configuring how the lein-tools-deps plugin should work with tools.deps#2018-05-1021:03alexmillershould be namespaced#2018-05-1021:03alexmillerwait, this is in project.clj?#2018-05-1021:04alexmillerI guess most stuff there is not namespaced#2018-05-1021:04alexmillersaying “lein” seems redundant#2018-05-1021:04alexmillerI guess I’d say :lein-tools-deps or :tools-deps#2018-05-1021:05dominicmNo harm in setting a new precedent.#2018-05-1021:06rickmoynihanyeah its in project.clj#2018-05-1021:06rickmoynihanbut thought it might be nice to set a precedent#2018-05-1021:06alexmiller:lein-tools-deps/config#2018-05-1021:09rickmoynihanyeah I had thought of that one too… I like that it nests config under it, but it gets wordy… though that’s not really an issue#2018-05-1021:09rickmoynihanit also matches the actual project name on github, and root namespace name#2018-05-1021:10rickmoynihanI think that’s definitely the most correct one#2018-05-1021:10rickmoynihanthanks#2018-05-1114:28ghadiI'm having trouble configuring an S3 private repo. To debug I've set -Srepro so that mavencentral & clojars are not seen by t.d.a, and I've set :mvn/repos to my own. For some reason I still get the error: cannot fetch the $dep from "central"#2018-05-1114:33ghadinever mind -- it's a bad error message. when all of the configured repos fail, it throws with one of them.#2018-05-1114:43ghadiTwo other misc issues:
cannot use override-deps to override from a git/sha to a local/root
:mvn/repos doesn't work transitively when pointing to a :local/root#2018-05-1115:04alexmiller-Srepro does actually include the install deps.edn btw#2018-05-1115:05alexmillerat the moment that is#2018-05-1115:07alexmillerre “cannot use override-deps to override from a git/sha to a local/root” - with same lib name? prob worth a ticket if so#2018-05-1115:07ghadiYes, same lib name#2018-05-1115:07ghadi👌:skin-tone-4:#2018-05-1115:07alexmillerre “:mvn/repos doesn’t work transitively when pointing to a :local/root” - I don’t understand what that means#2018-05-1115:08ghadiA => :local/root B => B has different :mvn/repos => C#2018-05-1115:09alexmillerok. that’s related to some of the other local dep issues that are already logged#2018-05-1115:12alexmiller@robert-stuttaford re twitter convo… current hack is:
{:deps {com.datomic/datomic.pro {:local/root "/path/to/datomic-pro-whatev.jar"}
com.datomic/datomic.pro.deps {:local/root "/path/to/dir-with-datomic-pom"}}#2018-05-1115:12alexmillerwhere the first pulls the jar and second treats the directory holding the pom file (which has to be named “pom.xml” as a local project to pick up the transitive deps)#2018-05-1115:13alexmillerthat’s the hack#2018-05-1115:13alexmillerhowever, I just landed a commit to make local jar deps use their embedded pom file to handle this without the hack#2018-05-1115:14alexmillerhttps://github.com/clojure/tools.deps.alpha/commit/de844ebcfe7787abe89f4f02b27abdbfdd5c7769#2018-05-1115:15alexmillerwill be in the next release#2018-05-1115:13ghadi(I don't think the transitive :mvn/repo thing should be supported, fwiw. Just logging the papercut)#2018-05-1115:14alexmillerwell then don’t log it :)#2018-05-1115:14alexmillerI’m kind of on the fence though - the equivalent will work if you’re using maven deps I think#2018-05-1115:14ghadiI'll file a ticket for the git -> mvn override-dep#2018-05-1117:42alexmiller@ghadi https://dev.clojure.org/jira/browse/TDEPS-51 is maybe sufficient for that#2018-05-1117:44ghadiyup, perfect#2018-05-1211:06robert-stuttafordthank you, @alexmiller! this is rad!#2018-05-1212:08mhuebertis there a simple way to specify a particular version of tools.deps.alpha itself (eg. to use last night’s release before it shows up on brew, or to point to a particular commit)?#2018-05-1212:10alexmillerNo, but I just did a release last night and there is a pending brew pr which should get merged today#2018-05-1212:11dominicmtime to bump packages#2018-05-1212:11alexmillerNew tools.deps should be available#2018-05-1212:13alexmiller0.5.435#2018-05-1213:08mhuebertah, brew has a nice way of pulling a PR. brew pull 27778 && brew upgrade clojure#2018-05-1217:32rickmoynihanI’ve made some changes to the lein-tools-deps README…
Any thoughts on how to improve it would be appreciated:
https://github.com/RickMoynihan/lein-tools-deps/tree/0.4.x#2018-05-1217:34dominicmhttps://github.com/RickMoynihan/lein-tools-deps/tree/0.4.x#why-not-use-boot-instead I don't know that you need to make this argument 🙂#2018-05-1217:39rickmoynihanI did consider not making it… as you’re probably a lein user if you’re reading it… But I figured new users might have been advised to use just one of the build tools; and be seeing this page, so thought it worth mentioning there are other choices too#2018-05-1217:39rickmoynihanor are you not a fan of boot? And would prefer everyone chose leiningen? 🙂#2018-05-1217:40dominicmThe opposite 😉 I'm the boot advocate at JUXT.#2018-05-1217:40rickmoynihanlol#2018-05-1217:41rickmoynihanthen is my comment fair?#2018-05-1217:48dominicmI think it is, other than perhaps:
> Leiningen projects are harder to turn into unique snowflakes, which might be better or worse for you.
For me the "snowflake" term has gained a disparaging connotation. But I understand what you're getting at, and I think it's mostly fair. We had issues with lein because it couldn't build sass & cljs at the same time, which made it a pain for users jacking in, but maybe we're in snowflake territory there?#2018-05-1218:06rickmoynihanYeah it is a slightly disparaging term; but I think it’s fair here - if you have a boot project it will often be pretty different to the next boot project; right?
More so than with lein projects.
I take your point that building sass/cljs simultaneously shouldn’t be such a unique a thing; but currently I think it is 😞 And yes if you want to do that boot is your best option; unless you want to build a lein plugin that essentially copies two other plugins entirely to compose them together on two futures 😞#2018-05-1314:52alexmillerNow available:
- tools.deps.alpha 0.5.435
- clj 1.9.0.381
- Support for Maven deps in authenticated repositories (TDEPS-9 - thanks Dominic Monroe!)
- In local jar dependencies like {:local/root “path/to.jar”}, if the jar has an embedded pom, it will be used to traverse and include transitive deps
- Use exec for final Java invocation in scripts (TDEPS-76 - thanks Mikhail Gusarov!)
- Convey lib map via Java system property - a first step towards add-lib and other future functionality
For Maven deps in authenticated repositories, existing Maven infrastructure is used to convey credentials via the ~/.m2/settings.xml:
<servers>
…
<server>
<id>my-auth-repo</id>
<username>zango</username>
<password>123</password>
</server>
…
</servers>
Then in your deps.edn include a repo with a name matching the <id>:
{:deps
{authenticated/dep {:mvn/version "1.2.3"}}
:mvn/repos
{"my-auth-repo" {:url ""}}}#2018-05-1322:11zentropeAre these release-notes/change-logs recorded anywhere? I see new homebrew versions every now and then and wonder what the changes are.#2018-05-1322:11zentropeIn other words, release notes of the whole thing CLI utility, not just individual libs?#2018-05-1323:31seancorfieldThe shell script etc's commit history is here https://github.com/clojure/brew-install/commits/1.9.0#2018-05-1323:32seancorfieldAnd the library is everything else https://github.com/clojure/tools.deps.alpha/commits/master#2018-05-1323:34seancorfield@zentrope ^ does that help?#2018-05-1323:40zentrope@seancorfield Yep! Thanks.#2018-05-1323:41zentropeI wish brew packages came with a change log of some sort built-in. Or brew info would show the latest changes. Mostly I don’t care, but sometimes it’s interesting.#2018-05-1400:25dottedmag@zentrope brew log -p clojure#2018-05-1400:26alexmillerBoth of those GitHub projects also have a CHANGELOG file too#2018-05-1400:28alexmillerThe scripts do install a man page for clj and clojure - I could probably add something there but I don’t know if that’s weird#2018-05-1400:28zentropeYes, I saw those. I glanced right over brew-install as a project before asking.#2018-05-1400:29zentropeYeah, easy change logs aren’t really part of the brew thing, it seems, outside of that @dottedmag just showed.#2018-05-1400:55alexmillerI added links from the man page to the changelogs for the next release#2018-05-1509:05shakdwipeeaHi guys, cross posting from #clojure
I am getting a dependency mismatch with s3. In my project I have specified dependency on com.amazonaws/aws-java-sdk-s3 {:mvn/version "1.11.261"}.
Also, I have a library whose transitive dependency include a different version of sdk . I have specified a exclusion for s3 as:
libB {:mvn/version "2.0.0-SNAPSHOT"
:exclusions [com.amazonaws/aws-java-sdk-s3]}
Now, when I do clj -Stree. I can see that only the correct version is being included.
But when I tried to find the location of sdk being included using
System.out.println(SDKGlobalConfiguration.class.getProtectionDomain().getCodeSource().getLocation()); as per https://github.com/aws/aws-sdk-java/issues/1071
I see that it is referring to the location of the jar of the transitive dependency which is the older version of aws sdk.
FWIW I am using the clj tool and deps.edn and the conflicting libraries are java based.
I am able to use the correct version from boot for the same tree.#2018-05-1522:29kennyWhere is the code for the clj cli tool located?#2018-05-1522:30ghadiIt's a bit oblique why, but it's here https://github.com/clojure/brew-install/blob/1.9.0/src/main/resources/clojure#2018-05-1522:31ghadiclj == rlwrap clojure basically#2018-05-1522:39kennyThanks#2018-05-1617:45kennyIs there a way to print the final deps.edn as data to stdout?#2018-05-1618:14alexmillercurrently, no#2018-05-1618:19alexmilleryou could use the output of -Sdescribe + a few calls in the api to replicate it#2018-05-1618:23kennyCool, figured it out. That option would be useful 🙂#2018-05-1618:27kennyFor those interested... Some psuedo code:
(defn full-deps-edn
[]
(let [config-files (:config-files (exec-sync-edn "clj -Sdescribe"))
deps "'{:deps {org.clojure/tools.deps.alpha {:mvn/version \"0.5.435\"}}}'"
code (str/join " " ['(require '[clojure.tools.deps.alpha.reader :as reader])
(list 'reader/read-deps config-files)])
cmd (str "clj -Sdeps " deps " -e '" code "'")]
(exec-sync-edn cmd)))
#2018-05-1722:06kennyHow do git deps work in a monolith type repo? It looks like I can only specify the root git URL.#2018-05-1722:16alexmillerYou can also use :deps/root to specify a relative dir path from the root#2018-05-1722:16kennyPerfect! I was looking here to better understand all the options I can pass but it looks like it's not quite up to date 🙂 https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/specs.clj#2018-05-1722:17alexmillerYeah those are a bit lagging#2018-05-1820:53kennyIs there a reason you shouldn't always use the -A CLI option?#2018-05-1820:55dominicmif the test alias uses test-runner, you may not want the main aspect of it 🙂#2018-05-1900:14seancorfieldThis is why I think it's better practice to have separate test and runner aliases 🙂#2018-05-1820:57kennyCould you clarify what you mean? What is test-runner? And what is the main aspect of it?#2018-05-1820:58dominicmSorry, I should be clearer, you may not want the :main-opts option to kick in.#2018-05-1820:59kennyAh that makes sense. Is that the only reason?#2018-05-1822:01hagmonkFun! clojure -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.6.0"} gorilla-repl {:mvn/version "0.4.0"}}}' -m gorilla-repl.core#2018-05-2013:11raymcdermottI’m trying to make use of add-lib using clojure 1.10-alpha4 with the prepl …#2018-05-2013:12raymcdermott@alexmiller is that combo possible?#2018-05-2013:12dominicm@raymcdermott what happened when you tried?#2018-05-2013:16raymcdermott:cause “Context classloader is not a DynamicClassLoader”#2018-05-2013:17raymcdermottbit weird though cos it partly works#2018-05-2013:20raymcdermottso it ‘works’ when evaluating something that is in the path … but that’s not so useful 😉#2018-05-2013:24raymcdermottbut I’m stabbing and poking around, so asking here to get a steer on whether it’s meant to be viable#2018-05-2013:44dominicm@raymcdermott My suspicion is that however the socket repl starts, doesn't cause a DynamicClassLoader to be run, I think you can insert one, but can't remember from the top of my head how that's done.#2018-05-2013:46alexmillerThat would be a good guess but I haven’t tried it#2018-05-2013:52dominicm@raymcdermott How are you running that clj file?#2018-05-2013:52raymcdermottlein run#2018-05-2013:53dominicmI think that uses aot-ish stuff, but I'm not certain.#2018-05-2013:55raymcdermottI’ll run it with clj and see if it makes any difference#2018-05-2013:56mfikes@raymcdermott when I hacked add-lib into ClojureScript, the hack needed to deal with the same issue: https://github.com/mfikes/clojurescript/commit/d68c9397599366777d9b322ec586fdd398302f25#2018-05-2013:57mfikesSee that bit where it forced the issue so that the classloader is a clojure.lang.DynamicClassLoader. This might not be the "correct" solution, but it worked.#2018-05-2013:58raymcdermottnice - I’ll check it out#2018-05-2014:00mfikesIf you end up needing to run that code for any reason to suss out what it is doing, https://gist.github.com/mfikes/a81ae3d006830dc34663f976aaf2291b#2018-05-2014:23alexmillerSetting the context classloader to a DCL should work#2018-05-2014:24alexmillerAlthough that is a per-thread solution#2018-05-2014:26raymcdermottok, so it’s the same whether I run it via CLJ or not … just thought I would check that out first#2018-05-2014:26raymcdermottDCL next - one thread is fine for now ( who needs more! 😉 )#2018-05-2014:40mfikesI'd cram it in early @raymcdermott (in a main if you can like in that ClojureScript hack), so any threads thereafter inherit it.
"When a Thread is created, it’s context classloader is set from the parent Thread." https://puredanger.github.io/tech.puredanger.com/2007/06/15/classloaders/#2018-05-2016:01alexmillerwow, old school reference :)#2018-05-2014:41raymcdermottparty-corgi#2018-05-2014:41raymcdermottwith DCL all works#2018-05-2014:41raymcdermottgood call @mfikes, will do - had it in the REPL code but will move it further out#2018-05-2015:49raymcdermottand - bonus - my code is now all set up with deps.edn and running with CLJ 😎#2018-05-2016:30robert-stuttafordwhat’s the correct way to force a SNAPSHOT download with clj / tools.deps?#2018-05-2017:10alexmillerThere isn’t a way at the moment - I have a ticket for that. You can just delete the version from your .m2 though and use -Sforce#2018-05-2018:04robert-stuttafordthanks Alex!#2018-05-2110:31borkdudeStandalone script using tools.deps: https://github.com/borkdude/balcony#2018-05-2110:34robert-stuttaford@alexmiller @dominicm thanks for TDEPS-9 - using Datomic PRO with clj is rad#2018-05-2110:35dominicm@robert-stuttaford I setup the JUXT website last night, was nice 🙂#2018-05-2110:35robert-stuttafordi recognise this may be a daft question, but is there perhaps a way to run two ‘mains’ at once from clj ? i see the docs say that it’ll run the last one encountered, but i’m hoping to get a boot-like thing going where i can run both figwheel-main and my 10-liner CIDER repl script in one process#2018-05-2110:36dominicm@robert-stuttaford Update to cider 0.17 and use -e before your normal -m#2018-05-2110:36robert-stuttaford-reads about -e-#2018-05-2110:36dominicmhttps://github.com/clojure-vim/vim-jack-in/blob/master/autoload/jack_in.vim#L56-L70 -e is eval#2018-05-2110:37dominicm^^ this might be useful to see what cider generates when it does jack-in, and hard-code it in 🙂#2018-05-2110:37dominicmbut you could implement cider-nrepl.main/init yourself.#2018-05-2110:37robert-stuttafordi’ve got a working cider script:
(require '[clojure.tools.nrepl.server :refer [start-server]]
'[refactor-nrepl.middleware :refer [wrap-refactor]])
(defn nrepl-handler []
(require 'cider.nrepl)
(ns-resolve 'cider.nrepl 'cider-nrepl-handler))
(def port 7800)
(spit ".nrepl-port" port)
(start-server :port port :handler (wrap-refactor (nrepl-handler)))
(println "Started nREPL on port" port)
(.addShutdownHook (Runtime/getRuntime)
(Thread. #( ".nrepl-port")))
#2018-05-2110:38robert-stuttafordi want to run it and figwheel-main together, in one process. right now i use two#2018-05-2110:38dominicmAre you using cljs? That doesn't look like it will work with cljs. Unless you have a separate nrepl for figwheel?#2018-05-2110:39robert-stuttafordhmm, i guess there’s no way around it; i’ll have to write a clj script that starts the nrepl-server and invokes the figwheel-main stuff afterwards#2018-05-2110:40dominicmYou can replace that whole file with:
clj -e "(require 'cider-nrepl.main) (cider-nrepl.main/init '[refactor-nrepl.middleware/wrap-refactor])"
#2018-05-2110:40dominicmI'm presuming that can go into the deps.edn somewhere.#2018-05-2110:45dominicm@robert-stuttaford Apparently not, but mains can run other mains.
(defn -main [& args]
(start-nrepl)
(apply clojure.repl/main args))
#2018-05-2112:07robert-stuttafordturns out the figwheel-main api is super simple, this was a total non-issue#2018-05-2112:07robert-stuttaford;; nREPL
(require '[clojure.tools.nrepl.server :refer [start-server]]
'[refactor-nrepl.middleware :refer [wrap-refactor]])
(defn nrepl-handler []
(require 'cider.nrepl)
(ns-resolve 'cider.nrepl 'cider-nrepl-handler))
(def port 7800)
(spit ".nrepl-port" port)
(start-server :port port :handler (wrap-refactor (nrepl-handler)))
(println "Started nREPL on port" port)
(.addShutdownHook (Runtime/getRuntime)
(Thread. #( ".nrepl-port")))
;; Figwheel
(require '[figwheel.main :refer [start]])
(start "dev")
#2018-05-2119:14jeff.terrellWith the new clojure CLI tool, can I have a standalone .clj script, i.e. apart from a src/ directory and without a corresponding deps.edn file, ideally invokable via a simple shebang line (`#!`) at the top of the file? If so, what would that shebang line be?#2018-05-2119:16dominicmYes#2018-05-2119:16dominicmDigging it up now#2018-05-2119:20dominicm@jeff.terrell
#!/bin/sh
"exec" "clojure" "-Sdeps" '{:deps {funcool/tubax {:mvn/version "0.2.0"}}}' "$0" "#2018-05-2119:20jeff.terrellImpressive. Thanks, @dominicm! I'm assuming the :deps are optional?#2018-05-2119:22dominicm@jeff.terrell what do you mean optional?#2018-05-2119:22jeff.terrellAs in, I don't need any deps. I can just leave that part out, right?#2018-05-2119:23dominicmOh, sure. In that case you can just do.
/usr/bin/env clojure
...
#2018-05-2119:24dominicmNo args = shebang. Args = exec#2018-05-2119:25jeff.terrellAh perfect. I did that initially, but I was getting confused because I expected clojure to automatically call my -main function. But I see now that it was working fine. Thanks for the help!#2018-05-2119:31dominicm@jeff.terrell I came up with it.#2018-05-2119:47jeff.terrellNice trick! Thanks again.#2018-05-2119:59jeff.terrellI added a PR for a FAQ entry about this on http://clojure.com:
https://github.com/clojure/clojure-site/pull/270#2018-05-2121:33jeff.terrellOK, followup question: how do I access command-line arguments to a #!/usr/bin/env clojure standalone script?#2018-05-2121:46mfikes@jeff.terrell They will be in *command-line-args*#2018-05-2121:47jeff.terrellThat's it. Thanks!#2018-05-2121:48mfikes@jeff.terrell If useful, some similar material to your FAQ is at http://planck-repl.org/dependencies.html under Shebang Deps#2018-05-2214:03borkdude@jeff.terrell @mfikes I used that approach extensively here: https://github.com/borkdude/balcony/blob/master/balcony.clj#2018-05-2214:04borkdudeI credited Planck for it.#2018-05-2215:02jeff.terrellThat is one freaky shell/Clojure chimera there, @borkdude. simple_smile Well done.#2018-05-2215:05jeff.terrellI think you should be able to replace the outer double-quotes after the semi-colons with single-quotes, so that you don't have to escape the inner ones.#2018-05-2215:06jeff.terrell(Bash deals with single quotes, and Clojure isn't going to care about them after the ;. That's my reasoning, anyway, but maybe I'm missing something.)#2018-05-2215:12alexmillerboot embeds a jar inside the shell script. I think that wins the chimera game for me.#2018-05-2215:13borkdude@jeff.terrell thanks, made the change#2018-05-2215:13mfikesThat one wins the "Hello my Baby" frog jumping out of the box award#2018-05-2307:16martinklepschThe error message when deps.edn’s parens aren’t balanced could be friendlier:
Error building classpath. EOF while reading
java.lang.RuntimeException: EOF while reading
#2018-05-2309:41alexmillerFile a ticket#2018-05-2314:49kennylein-tools-deps does not seem to be marking my src directory as a source root. I am using [lein-tools-deps "0.3.0-SNAPSHOT"] with :tools/deps set to [:system :home "/path/to/deps.edn"]. Has anyone else ran into this issue?#2018-05-2315:09mhuebert@kenny I have hit this just in the last couple of days, setting up a monorepo with a bunch of directories which each have their own deps.edn. I ended up adding "src" to :paths in each deps.edn#2018-05-2315:12kenny@mhuebert That is my exact situation and that solution did not work for me 😞#2018-05-2315:16mhueberthmm, I am trying to set up the repo on another machine now to do some more testing and getting a completely different error with lein-tools-deps
mattpro:chia MattPro$ lein pom
java.lang.NullPointerException: null
at clojure.tools.deps.alpha.util.maven$make_session.invokeStatic (maven.clj:105)
clojure.tools.deps.alpha.util.maven$make_session.invoke (maven.clj:102)
clojure.tools.deps.alpha.extensions.maven$eval851$fn__853.invoke (maven.clj:74)
clojure.lang.MultiFn.invoke (MultiFn.java:243)
clojure.tools.deps.alpha$expand_deps.invokeStatic (alpha.clj:170)
clojure.tools.deps.alpha$expand_deps.invoke (alpha.clj:152)
clojure.tools.deps.alpha$resolve_deps.invokeStatic (alpha.clj:215)
clojure.tools.deps.alpha$resolve_deps.invoke (alpha.clj:197)
lein_tools_deps.plugin$resolve_deps.invokeStatic (plugin.clj:78)
lein_tools_deps.plugin$resolve_deps.invoke (plugin.clj:69)
lein_tools_deps.plugin$middleware.invokeStatic (plugin.clj:93)
lein_tools_deps.plugin$middleware.invoke (plugin.clj:85)
#2018-05-2316:05mhuebert*figured out this was due to a too-old Leiningen version#2018-05-2315:19mhuebertthe line numbers on make_session make me think it is somehow not a recent version of tools.deps, though i just ran brew upgrade clojure#2018-05-2315:22alexmillerthis is a known issue with interpreting the paths for dependent repos#2018-05-2315:23alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-74#2018-05-2315:23alexmilleror at least that’s my guess as to what you’re seeing#2018-05-2315:23alexmillersetting :paths explicitly in each deps.edn should workaround it#2018-05-2315:25mhuebertthat workaround worked in my case#2018-05-2315:26mhuebertare local paths skipped intentionally during pom-generation or is that just WIP?#2018-05-2315:27mhuebertam doing a workaround currently, generating a project.clj for lein to use lein-tools-deps to generate a full pom which includes everything necessary for IntelliJ to figure out the project#2018-05-2315:34alexmillerthe first local path is added as the build source path, but I think that actually needs to be changed so it’s a resource path. I don’t think there’s actually a ticket right now about that#2018-05-2315:35alexmillerif you can file a ticket to remind me (or a patch!) that would be helpful#2018-05-2315:42mhueberthmm. why would it be a resource path?#2018-05-2315:42mhuebertI am comparing to what lein generates, it also makes <build><sourceDirectory>src</sourceDirectory> ... </build>#2018-05-2315:43mhuebertthat part seems to play well with intellij#2018-05-2315:46mhuebertSkipping paths: _build
Skipping coordinate: #:local{:root reactive}
Skipping coordinate: #:local{:root triple-db}
Skipping coordinate: #:local{:root view}
Skipping coordinate: #:local{:root gesture}
Skipping coordinate: #:local{:root style}
#2018-05-2315:48mhuebertI am wondering what can/should done about ^^these sorts of paths & local deps#2018-05-2315:50alexmilleroh, there is nothing that can be done about them - Maven has no concept of local deps#2018-05-2315:51alexmilleror git deps#2018-05-2315:51alexmillerso that’s never going to work#2018-05-2315:52alexmillerI guess you could write a custom Maven wagon and use that, but I’m not going to do that#2018-05-2315:57mhuebertit looks like lein pom uses a plugin (`build-helper-maven-plugin`) to add the local sources for those:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>_build</source>
<source>reactive/src</source>
<source>triple-db/src</source>
<source>view/src</source>
<source>view/test</source>
<source>gesture/src</source>
<source>style/src</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
#2018-05-2315:58mhueberthttps://github.com/technomancy/leiningen/blob/master/src/leiningen/pom.clj#L258#2018-05-2316:00mhueberthttps://www.mojohaus.org/build-helper-maven-plugin/usage.html#2018-05-2316:01alexmilleryeah, I’m not going to use that#2018-05-2316:01alexmilleryou can add these to your pom manually if you like. -Spom will sync an existing pom, just updating the rest of it#2018-05-2316:03mhuebertok. at some point maybe intelliJ/cursive will support deps.edn directly#2018-05-2316:47alexmillerI think he’s started on that#2018-05-2319:46hagmonkANN: I just updated depify so that it can work from a deps.edn alias, if anyone is interested: https://github.com/hagmonk/depify#2018-05-2408:59mhuebert@kenny I just found something that may be related to your issues yesterday. In my mono-repo-of-`:local/root`-projects, if I change one of the deps.edn files inside of a :local/root directory, tools.deps does not recognize the change, and uses its previously cached classpath. I have to make some minor modification to the root deps.edn file (like, add a space somewhere) and then it invalidates the cache. So you may have configured everything correctly and clj was just using an old classpath. Can test for this by running clj -Spath before/after changes to see when it uses the cache vs. when it generates a fresh one.#2018-05-2409:08mhuebert@alexmiller would it be worth creating an issue for this? I am not sure exactly where/how cache invalidation is handled#2018-05-2412:12alexmillerI have an issue for it#2018-05-2412:15alexmillerActually looks like maybe I never filed a public issue but I have it in my internal list#2018-05-2412:15alexmillerWorkaround is to use -Sforce#2018-05-2415:54borkdude-e, --eval string Eval exprs in string; print non-nil values
is it possible to turn off the printing?#2018-05-2416:02borkdudenever mind#2018-05-2418:47raymcdermottwhen using add-lib I sometimes get errors when adding via a git SHA …. Manifest type not detected when finding deps#2018-05-2418:47raymcdermotthere is the longer stack trace#2018-05-2418:48ghadiraymcdermott you may be add-lib'bing a git repo that has no deps.edn in it#2018-05-2418:48raymcdermotta ha!#2018-05-2418:48raymcdermott@ghadi I did not realize that was a pre-requisite#2018-05-2418:49raymcdermottpretty sure that is the case then#2018-05-2418:49ghadiYeah. tools.deps cannot infer third-party dependencies without some sort of manifest file#2018-05-2418:50ghadiright now deps.edn and pom.xml are supported#2018-05-2418:50raymcdermottok, cool so that’s why Maven deps are safer#2018-05-2418:50raymcdermottfor the time being at least 😉#2018-05-2418:52raymcdermottat least I know which repos I can demo now#2018-05-2418:55raymcdermottoff the wall question … has tools.deps been tried out on graal?#2018-05-2418:58alexmillerNot to my knowledge#2018-05-2418:59alexmillerI suspect it’s bounded by network io usually#2018-05-2419:00raymcdermottI was thinking that it probably couldn’t change the classpath#2018-05-2419:01ghadiyeah, it cannot. add-lib will not work over there.
Does the REPL work in Graal?#2018-05-2419:01raymcdermottI’m guessing that an app can still expose a socket-server#2018-05-2419:02ghadi(I'm assuming you mean the Graal AOT binary part, not just running Graal through JVMCI)#2018-05-2419:02ghadithe repl (whether socket server or otherwise) generates classes, which is incompatible with Graal's AOT native images.#2018-05-2419:03ghadiBut all that should work properly through Graal non-AOT (the JVMCI)#2018-05-2419:03raymcdermottthat sounds legit…#2018-05-2419:04raymcdermottso that means - in theory at least - that you can call another language via the REPL#2018-05-2419:07raymcdermottthey have a polyglot shell so I’m guessing yes at this stage#2018-05-2419:08raymcdermottalthough it seems to switch languages rather than offer FFI#2018-05-2419:09raymcdermotthmmm this seems to be the thing#2018-05-2419:09raymcdermotthttp://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Context.html#2018-05-2419:10raymcdermottok anyway … that’s a different channel#2018-05-2419:11raymcdermottthanks for the help @ghadi#2018-05-2419:51raymcdermottbtw I made a small demo app with the pre-reqs in place for add-lib#2018-05-2419:51raymcdermotthttps://github.com/raymcdermott/add-lib-demo-app#2018-05-2419:52raymcdermottI have proven it with my REPL so it might work elsewhere 🙂#2018-05-2518:12kennyIs there a reason the -Spom option does not offer any way to configure the generated pom (i.e. project name and group id are the directory name and the version is hardcoded to 0.1.0)?#2018-05-2518:15alexmiller-Spom syncs an existing pom, so run it, then modify the pom. subsequent runs will keep that stuff.#2018-05-2519:46kennyIf I run this
clj -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.9.0"}}}' -Spath
In a directory containing a deps.edn that looks like this:
{:deps {org.clojure/clojure {:mvn/version "1.8.0"}
org.clojure/core.async {:mvn/version "0.4.474"}}}
I get a classpath containing core.async. Why does that happen? I thought -Sdeps was supposed to be the final deps.edn file.#2018-05-2520:03hiredmanhttps://clojure.org/reference/deps_and_cli#_classpath_construction#2018-05-2520:04hiredmanit looks like dep.edns are merged, and -Sdeps is the final value merged#2018-05-2520:05kennyOh, you're right. Hmm, ok. It'd be nice to have a way to easily reproduce issues by explicitly passing in the one and only deps edn.#2018-05-2521:41seancorfieldThere's an argument with -Srepro that it should ignore everything except the current directory's deps.edn -- on the grounds that it would make a project "reproducible". As it is, ignoring the install directory with -Srepro means that you need extra stuff in your project deps.edn that you wouldn't normally need (`:paths`, repos?) which means you can't just take an arbitrary project's deps.edn file and run it with -Srepro unless that's how the author expected it to be run.#2018-05-2521:47kennyI was asking for a directory independent way of running clj. That topic is somewhat related but still different.#2018-05-2521:56seancorfieldYeah, having more control over which of the four deps locations were consulted would be very nice.#2018-05-2521:57seancorfieldAnd your scenario -- ignoring the current directory's deps.edn -- is a good use case that it would be nice to see supported...#2018-05-2615:41renewdoitIs there any idea to download deps in parallel to speed up?#2018-05-2616:59alexmillerIt’s something I have thought about. It’s a little tricky the way things are set up right now but def doable#2018-05-2809:01borkdudewithout Srepro deps aren’t downloaded but with it they are?
clojure -Srepro -Spath
I need the path but also the effect of deps being downloaded, but nothing else. What’s the best option for this?#2018-05-2809:02borkdudethis is excellent for lumo/planck btw, very fast#2018-05-2813:43alexmillerrepro should not affect downloading. Are you just seeing m2 caching?#2018-05-2813:58borkdude@alexmiller when I e.g. delete tools.cli from my .m2 repo, only -Spath does not force downloading, but it works with -Srepro. What should the behavior be?#2018-05-2814:07alexmillerOh, you’re misinterpreting stuff due to classpath caching#2018-05-2814:08alexmillerNormally, if there is a cached classpath, it’s used (and the actual paths aren’t checked, so it won’t notice you deleted from m2). But if there is no cached classpath it will download#2018-05-2814:08borkdudeif I can set M2_HOME somehow, I could try to reproduce. Don’t know how to do that with clj#2018-05-2814:09borkdudeah, that explains it#2018-05-2814:09alexmillerWith repro, I think it does not use the cached classpath#2018-05-2814:09alexmillerSo forces download #2018-05-2814:09alexmillerYou can use -Spath -Sforce to do what you want#2018-05-2814:09borkdudeso in normal circumstances ( I don’t delete something from .m2 manually ) all should work according to plan#2018-05-2814:10alexmillerYes, in the normal circumstances things work normally#2018-05-2814:11alexmiller;)#2018-05-2814:11borkdudeyeah, but -Sforce takes longer, that’s not preferred for planck or lumo, so I’ll go for just -Spath#2018-05-2814:11alexmillerWe actually have a ticket open for noticing broken paths #2018-05-2814:11borkdudehehe, yes, I mean, there is no logic to actually check for the existence of jars after classpath caching#2018-05-2814:12borkdudeso, I understand now, thanks#2018-05-2814:12borkdudeside question, is there a way to change which .m2/repository to use with clj?#2018-05-2814:12alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-70#2018-05-2916:55rickmoynihanMerely as an extra data point I ran into this issue yesterday because I’d blown an .m2 dep away whilst deving/testing some CI stuff. I realised I also needed to blow .cpcache away too to get the effect I wanted; but either a good message, or re-resolving the missing deps would’ve been super nice. It could easily have been a hard error to diagnose.#2018-05-2916:56alexmilleryeah, I’ve been there (even knowing how it works)#2018-05-2814:13alexmillerYes, you can set :deps/local-repo in deps.edn map#2018-05-2814:13borkdudeah, that’s actually the issue yes. well, if it takes extra time it would be worth considering not doing it… or with another option#2018-05-2814:13borkdudeif it’s a couple of ms, it would be ok#2018-05-2814:14alexmillerYeah I haven’t done any eval on it yet#2018-05-2814:14alexmillerTBD #2018-05-2814:15alexmillerThere are also cases where you might expect to have a missing path dir#2018-05-2814:15alexmillerAnd would want that to be ok#2018-05-2814:16borkdudeI have this in deps.edn:
{:deps/local-repo "/tmp/repro"}
and do this from /tmp/repro:
clj -Sforce -Sdeps '{:deps {org.clojure/tools.cli {:mvn/version "0.3.7"}}}' -e "(require '[clojure.tools.cli])"
it doesn’t seem to use the local dir#2018-05-2814:28borkdudejust upgraded to deps.edn file#2018-05-2814:31borkdudeah, it was :mvn/local-repo#2018-05-2816:44alexmillerOh sorry, yeah that :)#2018-05-2901:00hagmonkDoes anyone have a deps.edn snippet or advice on how to get a browser based CLJS app running, with an nREPL that Cider is able to reach? Basically like the new CLJS quickstart guide, except with the REPL in emacs instead.#2018-05-2903:39richiardiandrea@hagmonk the new cider should inject all the necessary deps as long as you have a deps.edn in the root folder#2018-05-2903:39richiardiandreaAlso better ask this in #cider#2018-05-2903:54hagmonk@richiardiandrea to rephrase my question so that it's not being confused with something cider specific: does anyone have a deps.edn snippet or advice on how to get a browser based CLJS app running, using cljs.main, such that it starts an nREPL server rather than an interactive terminal REPL?#2018-05-2904:04richiardiandreaDo not have that, but it would be a very small variation of this gist: https://gist.github.com/arichiardi/8ad47f5edf185876e82015739f038a22
Still believe some folks can help you in #cider simple_smile#2018-05-2907:52delaguardoHi there! I have a question about :default-deps. Does anybody have a good example how to use it? Official documentation tells it provides a set of default coordinate versions to use if no coordinate is specified. But how I can not specify coordinate versions for the :deps or :extra-deps?#2018-05-2907:52delaguardoHi there! I have a question about :default-deps. Does anybody have a good example how to use it? Official documentation tells it provides a set of default coordinate versions to use if no coordinate is specified. But how I can not specify coordinate versions for the :deps or :extra-deps?#2018-05-2911:54alexmillerThis area is a little incomplete and there is not really a great story around how to use it yet#2018-05-2912:18delaguardoThere is my case if you are interested: I’m trying to create “monorepo” with root deps.edn and a few “sub-projects” with their own local deps.edn. I would like to have a list of all used dependencies with their coordinates as a :default-deps in root deps.edn. All local deps.edn files should have only a list of dependencies without coordinates specified.#2018-05-2913:22alexmilleryeah, this is in line with what we intended but there needs to be another round of thinking on how to make this a reality#2018-05-2915:06delaguardo👍#2018-05-2910:53shakdwipeeaHi guys, One of my transitive deps is trying to pull javax.activation and running into this error:
Error building classpath. Could not find artifact activation:activation:jar:1.0.2 in central ()
org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact activation:activation:jar:1.0.2 in central ()
Turns out the jar is not available in maven but it is available in one of the adobe public repos.
So I added
{:mvn/repos {"adobe-public" {:url ""}}}
Now this only works if javax.activation is a direct dependency but not when it is a transitive dep.
Also, the jar is available in maven local but I am still running into this.#2018-05-2910:58dominicmI've seen some funky poms where the pom actually refers to other repositories and those are transitively picked up.#2018-05-2911:56alexmiller@shakdwipeea are you getting to these transitive deps through a maven dep or a local/git dep?#2018-05-2912:00shakdwipeea@alexmiller It is coming through a git dep. I also tried with https://jitpack.io to use it as a mvn dep but that also didn't work. The funny thing is that the required dep is present in my maven local but still this fails.#2018-05-2912:01shakdwipeeaFWIW the dep is a java project and uses maven.#2018-05-2912:14alexmillerYeah this is a known issue with git and local deps not picking up the repos in the pom#2018-05-2912:17alexmillerOr at least I thought it was, I don’t see a ticket for it. I’d be happy to have a ticket with more specific details so I can take a look#2018-05-2915:23shakdwipeeaI'll try to create a minimal repro and file a ticket.#2018-05-2915:44alexmillerthx#2018-05-3007:05shakdwipeeaI tried to reproduce the issue on a new repo. https://github.com/shakdwipeea/tdeps-transitive-repo-test Now when you clj -A:test-git it fails with the message that
`
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact activation:activation:jar:1.0.2 in central ()
The same deps works with the boot file that is present in the repo. The interesting thing is that
when I do boot show -d, I get
[com.github.shakdwipeea/smtp "0.3.1"]
└── [javax.javaee/javaee "6.0-alpha-1"]
├── [javax.activation/activation "1.1-rev-1"]
...
This uses the version 1.1-rev-1 which is available in the maven repo.
The 1.0.2 version should not even be fetched.
I am not sure if I am doing sth wrong, so wanted to consult before opening a ticket.#2018-05-2918:13hagmonk@alexmiller how does one contribute to the tools.deps.alpha github wiki? I wanted to put a link to depify, and @seancorfield's clj-new.#2018-05-2918:14alexmillerit’s a wiki - anyone should be able to edit#2018-05-2918:15hagmonkCool! I did see the edit button but didn't want to boldly assume :)#2018-05-2918:17seancorfield@hagmonk Thank you in advance for adding a link to clj-new 🙂#2018-05-2918:21hagmonkDone! I'm sure there are more that could be added ...#2018-05-3011:28borkdude@shakdwipeea what if you move the deps to a toplevel :deps?#2018-05-3011:29shakdwipeeaI had tried that.. Same result.. Made aliases to test out the two conditions separately#2018-05-3113:05rickmoynihanFYI, we’ve just merged :resolve-aliases support in to lein-tools-deps. Currently available only on the 0.4.x branch:
https://github.com/RickMoynihan/lein-tools-deps/blob/0.4.x/example/project.clj#L20#2018-05-3116:03kennyHow do I load a Clojure file on the classpath from the CLI? I tried doing clj -i my.ns and I get a FileNotFound exception.#2018-05-3116:05kennyI guess I could just call load in -e. Is there an easier way though?#2018-05-3116:07ghadiclj -i foo.clj should work -i expects a file name, not a namespace#2018-05-3116:22kennySay I have a monolith type git repo that looks something like this:
mylib1
mylib2
mylib2 has a deps.edn files that looks like this:
{:deps {mylib1 {:local/root "../mylib1"}}}
Is it possible to use mylib2 as a git dep (`{:git/url "https://github.com/..." :sha "..." :deps/root "mylib2"}`) from the CLI and have mylib1 dependency be resolved using the :local/root path and the same :sha used previously?#2018-05-3116:28kennyOr perhaps even changing mylib1 coordinate to be {:git/url "" :deps/root "mylib1"} and the :sha would be automatically set to the same :sha used for the the git URL ... that is already on the classpath.#2018-05-3116:32alexmillerIn short, no - git libs are only resolved from git#2018-05-3119:54kennyIf I have foo.clj:
(ns foo)
(defn bar
[& args]
(prn *command-line-args*))
And run:
clj -i foo.clj -e '(foo/bar)' asd
I get a FileNotFoundException.#2018-05-3119:55kennyI assume this is because it is expecting the last param to be a path. Why does it require that?#2018-05-3119:58ghadi@kenny https://github.com/clojure/clojure/blob/master/src/clj/clojure/main.clj#L388-L417
hope that helps#2018-05-3119:58ghadiyes the last param is a path#2018-05-3119:59kennyRight. Why does it require that?#2018-05-3120:02zentropeAre relative paths wonky for deps? For instance, if you local/root a deps.end which in turn local/root’s a dep (something like “../../../whatever”) is the “../../..” going to mess things up if the top level isn’t in a “../../../” relationship to that project?#2018-05-3120:08zentropehttps://dev.clojure.org/jira/browse/TDEPS-74#2018-05-3120:03seancorfield@kenny Use - to introduce the arguments: clj -i foo.clj -e '(foo/bar)' - asd#2018-05-3120:03zentropeHm. Overriding it in the “current” deps.edn solves it.#2018-05-3120:04kenny@seancorfield The program doesn't exit then.#2018-05-3120:05ghadi@kenny what are you trying to achieve?#2018-05-3120:05seancorfieldAh, yes, you'd have to send it empty input 🙂#2018-05-3120:05kennyTo evaluate the function in -e.#2018-05-3120:05ghadithere may be a more straighforward path.#2018-05-3120:05kenny(foo/bar)#2018-05-3120:06seancorfieldYou can only get *command-line-args* if you have main-opts#2018-05-3120:06ghadiyou can -e "(require 'foo)" -e "(foo/bar)"#2018-05-3120:07kennyI also want *command-line-args* 🙂#2018-05-3120:07ghadioh -- yeah you can't.#2018-05-3120:08ghadiOr, this (cue eye roll):
clj - <<EOM
(require 'foo)
(foo/bar)
EOM
#2018-05-3120:08ghadi(In bash or zsh)#2018-05-3120:08seancorfieldRename bar to -main and then clj -i foo.clj -m foo asd#2018-05-3120:08ghadi^^^#2018-05-3120:08seancorfield(! 633)-> cat foo.clj
(ns foo)
(defn -main
[& args]
(prn *command-line-args*))
Thu May 31 13:08:39
(sean)-(jobs:0)-(~/clojure/kenny)
(! 634)-> clj -i foo.clj -m foo asd
("asd")
#2018-05-3120:08kennyBleh. And I can't rename because -main is used for something else.#2018-05-3120:09kennyI also may want to call different functions besides -main.#2018-05-3120:09seancorfieldAdd another namespace for this entry point?#2018-05-3120:10kennyPerhaps. But I may have 50 different functions that I'd like to call. I don't want 50 namespaces to simply call a function.#2018-05-3120:11seancorfieldOK, how about this: (! 638)-> cat foo.clj
(ns foo)
(defn -main [& args] (println "I am foo's -main"))
(ns foo.main)
(defn -main
[& args]
(prn *command-line-args*))
Thu May 31 13:10:56
(sean)-(jobs:0)-(~/clojure/kenny)
(! 639)-> clj -i foo.clj -m foo
I am foo's -main
Thu May 31 13:11:04
(sean)-(jobs:0)-(~/clojure/kenny)
(! 640)-> clj -i foo.clj -m foo.main asd
("asd")
#2018-05-3120:11seancorfield(multiple namespaces in one file 🙂 )#2018-05-3120:12kennyAh, interesting.#2018-05-3120:12kennyStill feels like a bit of a hack but that could work. It seems like it'd be useful to have a CLI option to execute a function besides -main.#2018-05-3120:13seancorfieldWe have a job script at work whose first argument is a namespace-qualified symbol and it calls that from -main. So you'd say clj -i job.clj -m job job/foo asd#2018-05-3120:14seancorfieldRelying on dynamic require and resolve#2018-05-3120:14kennyYeah, I think I like that option more.#2018-05-3121:43kennyShould :override-deps work to change an artifact's coordinate from Git to local?#2018-05-3121:53kennyIt doesn't seem to work. Example a/deps.edn:
{:deps {org.clojure/tools.cli {:mvn/version "0.3.7"}}}
b/deps.edn:
{:deps {a {:git/url ""
:sha "7b8e5c33588a94c1318e225dc9fe655599216370"}}
:aliases {:dev {:override-deps {a {:local/root "../a"}}}}}
Result of clj -A:dev -Sforce -Spath:
src:/home/kenny/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/home/kenny/.gitlibs/libs/a/a/7b8e5c33588a94c1318e225dc9fe655599216370/src/main/clojure:/home/kenny/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/home/kenny/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar
That classpath does not contain tools.cli as required by the overridden a dependency.#2018-05-3122:14alexmillerNo, there is a ticket for this right now#2018-05-3122:17kennyhttps://dev.clojure.org/jira/browse/TDEPS-51. Is the plan to allow this behavior or throw?#2018-05-3122:38alexmillerTo allow#2018-06-0115:29kennyAny way to use aliases defined in a library that is in your :deps?#2018-06-0115:46alexmillerno#2018-06-0115:34rickmoynihanFYI :aliases, :resolve-aliases and :classpath-aliases are implemented on 0.4.x branch of lein-tools-deps… just need to kick the tyres a bit and add some documentation before merging to master#2018-06-0120:10hypirionIs there any way to compile or refer to Java files with deps.edn, or do I have to do that in a separate project?#2018-06-0120:11dominicm@hypirion Separate#2018-06-0120:14hypirionOkay, I guess that is by design? It's a bit iffy if I just want to improve the performance of a component in the system by converting it to Java.#2018-06-0120:31alexmillerin scope: a) make classpaths and b) launch Clojure programs#2018-06-0120:31alexmillerthere are a vast array of other “build” activities you may wish to do#2018-06-0120:31dominicmI have thought about writing something which scans for .java rules in directories on the classpath and builds them.#2018-06-0120:32alexmillerwell there are about 10 programs that do that already#2018-06-0120:32dominicmThere are?#2018-06-0120:33alexmillerSure, Ant, Maven, Leiningen, Boot, Ivy, Grape,#2018-06-0120:33alexmillerand on and on#2018-06-0120:34dominicmSure. They're a bit different. By default they don't expect to compile dependencies, eg local and root dependencies#2018-06-0120:34alexmillersome yes, some no#2018-06-0120:34alexmilleryou can easily coax Ant or Ivy into this#2018-06-0408:22borkdudeI’m looking into the nix package manager right now and wonder if classpath caching was inspired by this maybe?#2018-06-0409:40mkvlr@borkdude as someone without fear of parens guix might also be of interest to you. It’s basically nix written in scheme.#2018-06-0409:41borkdudecool#2018-06-0410:31robert-stuttafordwhat are the rules for downloading snapshots? i have two extra :mvn/repos defined (Datomic’s, and our own S3 one), and for some reason, tools.deps is failing on figwheel.main’s dep because it’s not attempting it on the one repo that has it - http://clojars.org.#2018-06-0410:32robert-stuttafordgosh, nevermind. i had a non-standard space after the dep name, which caused it to be included in the dep name 🙈#2018-06-0410:39dominicmthat's horribly awesome.#2018-06-0411:13borkdudemaybe write a patch for Clojure to treat that like the other whitespace 😛#2018-06-0411:13dominicmHow will I drive my co-workers crazy then?#2018-06-0415:51dominicmI'm having an issue I don't really understand, I'm trying to depend on jnr-posix, with lein all is happy, and with clojure tools the same snippet doesn't work.
com.github.jnr/jnr-posix {:mvn/version "3.0.42"} is the version.
user=> (import
#_=> (jnr.posix POSIXFactory POSIXHandler))
jnr.posix.POSIXHandler
user=> (POSIXFactory/getNativePOSIX)
#object[jnr.posix.LinuxPOSIX 0x6dd5e5ec "
Is the snippet that does work in lein, but not in clojure cli tools.
I even tried doing lein deps :tree and using those dependencies in deps.edn:
{com.github.jnr/jnr-posix {:mvn/version "3.0.42"}
com.github.jnr/jnr-constants {:mvn/version "0.9.9"}
com.github.jnr/jnr-ffi {:mvn/version "2.1.7"}
com.github.jnr/jffi {:mvn/version "1.2.16"}
com.github.jnr/jnr-x86asm {:mvn/version "1.0.2"}
org.ow2.asm/asm-analysis {:mvn/version "5.0.3"}
org.ow2.asm/asm-commons {:mvn/version "5.0.3"}
org.ow2.asm/asm-tree {:mvn/version "5.0.3"}
org.ow2.asm/asm-util {:mvn/version "5.0.3"}
org.ow2.asm/asm {:mvn/version "5.0.3"}}
But that hasn't resolved the issue either.#2018-06-0415:55dominicmLooks like in the deps.edn version, ( "jni/x86_64-Linux/libjffi-1.2.so") doesn't resolve.#2018-06-0415:55dominicmOH, I see. It's because jffi has a native classifier.#2018-06-0415:57dominicmTDEPS-50 and 12. No fun being here 😛#2018-06-0415:57dominicmIs there any known workarounds?#2018-06-0416:10alexmilleryou could pull the other classifiers in as local jars#2018-06-0416:11alexmiller{:deps {madeup/dep {:local/root "path/to/the.jar"}}}#2018-06-0416:19dominicm@alexmiller this is in pack, I'd like to keep the repo small (no jars in the repo). But I've done that into my maven repo for now, only I need to run the tests, right?#2018-06-0416:22dominicmTrying to write some tests for pack, which involves changing the jvm directory in order to have :local/root "../blah" work correctly.#2018-06-0416:52alexmillerI don’t understand what you’re saying, sorry#2018-06-0417:17dominicm@alexmiller I'm changing the cwd of the process so that relative paths work in deps.edn files.#2018-06-0417:18dominicmSo that I can write tests for pack#2018-06-0417:42dominicmhttps://www.engineyard.com/blog/bats-test-command-line-tools maybe this would have been less effort...#2018-06-0422:31kennyI'd like to make a script to run tests and launch that script with clj. Something like clj -A:test -m test-runner. In order to run tests, you need to know which namespaces you want to test. To determine the namespaces to test you need to find all namespaces given some file system paths. I'd like to be able to figure out the file system paths from the super set of :paths and :extra-paths in your deps config files given the aliases you passed to the clj command. I can't see a way to get the aliases you used to launch the clj command. Any ideas?#2018-06-0422:39seancorfield@kenny Have you looked at Cognitect Labs' test-runner project?#2018-06-0422:39seancorfieldhttps://github.com/cognitect-labs/test-runner#2018-06-0422:42kennyOh cool! Seems like that project takes a different approach by defaulting to test directory otherwise you need to explicitly specify the directory for your tests. I suppose that'd work.#2018-06-0422:44kennyI still think it would be useful to introspect the clj props used to launch the JVM.#2018-06-0422:45seancorfieldSee https://github.com/seancorfield/dot-clojure for an example of using test-runner where the tests could be set by an alias (I specifically run tests with clj -A:test:runner -- where :test can vary by project)#2018-06-0422:46seancorfieldHere's an example of that test-runner used with multi-version testing https://github.com/clojure/java.jdbc/blob/master/deps.edn#2018-06-0422:47seancorfieldhttps://github.com/clojure/java.jdbc/blob/master/run-tests.sh#L47#2018-06-0422:48seancorfield> I can't see a way to get the aliases you used to launch the clj command. Any ideas?
I'm not sure why you'd need/want this?#2018-06-0422:49seancorfieldYou can't do anything with those aliases...#2018-06-0422:49kennyMakes sense.#2018-06-0422:50kennyThere are lots of things I can do with the aliases. I can use them to determine information about the deps.edn file. I can launch sub-processes with the same classpath as the parent process.#2018-06-0422:52seancorfieldSo this isn't a real world need, just a "what if"?#2018-06-0422:52kennyIt is real world because that's the exact problem I'm running into right now 🙂#2018-06-0422:53seancorfieldCan you elaborate? What, specifically, are you trying to do where knowing the actual aliases used would be a solution?#2018-06-0422:55kennyA simple example would be to get a set of all :paths and :extra-paths used to launch the JVM via clj.#2018-06-0422:56seancorfieldYou can already get the classpath in any JVM program... and then you can walk that to see which are directories and which are files...#2018-06-0422:56kennyDoesn't work with local deps 😉#2018-06-0422:56kenny:local/root#2018-06-0422:57kennyThe set of all directories on the classpath may be a super set of :paths, :extra-paths, and other directories with no way to differentiate the three.#2018-06-0422:58seancorfieldAnd you need to tell them apart because... why?#2018-06-0422:58kennyDifferentiate files in your current project from dependencies.#2018-06-0422:58kennyI simply want a list of files in :paths and :extra-paths.#2018-06-0422:59seancorfieldSorry, I'm still not understanding the actual problem you're trying to solve here. I'll just give up and go away.#2018-06-0423:01kennySorry, I should have started with the most precise, minimal problem.
I'd like to get a list of files within the :paths and :extra-paths for the clj launched JVM within the JVM.#2018-06-0423:02seancorfieldYes, you keep saying that, but I still don't understand what problem you are trying to solve.#2018-06-0423:07kennyIt could be anything - this will be used for general operations over your project files. For example, I may want to run a formatter over all my project files. As done in the test-runner you sent, I could require the user to explicitly pass the directory and default to something like ["src" "test"]. It seems preferable, however, to default to the current set of files within :paths and :extra-paths because these are the files you are currently working with.#2018-06-0423:13seancorfieldWhy not use the current working directory where the <insert general tooling> was invoked, and walk the classpath for paths that include that directory as a prefix? Then you have something that is general and could be invoked via clj, lein, or boot.#2018-06-0423:14seancorfield(with an option to specify paths as -main arguments)#2018-06-0423:16kennyBut why require the user to specify a prefix when they already included the directories they want via aliases?#2018-06-0423:16seancorfieldRe-read what I suggested.#2018-06-0423:16seancorfieldThe user would not have to specify a prefix.#2018-06-0423:17kennyOh I see.#2018-06-0423:17seancorfieldThen the tooling you produce really is general rather than being hardwired to some implementation artifact of how clj happens to work today.#2018-06-0423:17kennyI don't believe that is general though. You can specify a path via an alias that is not in the parent path.#2018-06-0423:19seancorfieldIf :paths or :extra-paths end up pointing to directories outside the project where you invoked clj (and your imaginary tooling), I think most people would be surprised that it performed operations on those files, rather than what is local inside the current project folder...?#2018-06-0423:21kennyPerhaps. But they also explicitly included that as a :path instead of a local dependency so they probably have a reason for doing.#2018-06-0500:36alexmillerthis should be considered highly unusual and possibly even a bug#2018-06-0423:22seancorfieldI think you're trying to solve a non-problem and making your tooling set up more complicated (and more brittle) than it needs to be.#2018-06-0423:23seancorfieldSolve the simple, obvious use case first -- using just the current working directory and the classpath, which is the general tool-agnostic case -- and then see if any of your users really do things differently.#2018-06-0423:24kennyThat's fair 🙂#2018-06-0423:24seancorfield(I've learned this the hard way with boot-tools-deps -- with hindsight, I would have done that very differently)#2018-06-0423:25kennyWould you prefer the test-runner project default to directories on the classpath that are prefixed with your current directory or simply the "test" directory?#2018-06-0423:27seancorfieldFor a test runner, it makes sense to provide an external, tool-agnostic way to specify where to find tests -- since you already know the source code is going to be on the classpath (as well as the test code).#2018-06-0423:29seancorfieldFor example, in simple Clojure projects, src and test are the two top-level folders. In mixed clj/cljs projects, you often have src/clj, src/cljc, src/cljs (and similarly for test) so specifying just what should be run on the JVM is already more complex. And in Maven-style projects, tests are under src/test -- where source code is under src/main (and those tend to have <language> as the next level and then the actual code tree).#2018-06-0423:33kennyMakes sense. Thanks for the direction.#2018-06-0604:05cflemingI’m looking into integrating deps support into Cursive. I have some questions about how deps is expected to be invoked for this use case. I’m assuming for the moment that the user must have the command line tools installed, and will have to configure the path to the clojure command.#2018-06-0604:06cflemingOnce I have that, I can then execute with either -Sverbose or -Sdescribe, and get some information about the environment, and I can get a classpath.#2018-06-0604:06cflemingBut I can’t see a way with the command line tool to get path information like source and test paths.#2018-06-0604:08cflemingI think I’ll have to be using tools.deps.alpha to read e.g. deps.edn files, but I also can’t see a good way to find either the uberjar from the local install, or get the tools.deps.alpha version that I would need to download it myself and use it as a library.#2018-06-0604:09cflemingIs anything provided for this kind of use case?#2018-06-0604:10cflemingAlso, I’m assuming that deps doesn’t provide any help for downloading source artifacts of dependencies?#2018-06-0604:19cflemingI just tried clojure -Sdeps '{joda-time/joda-time {:mvn/version "2.9.7" :classifer "sources"}}' -Spath but no luck.#2018-06-0605:29seancorfield@cfleming Not sure whether you can rely on it "forever" but clojure -A:deps brings in the tools.deps.alpha version...#2018-06-0605:33seancorfield(and, yes, the tools.deps.alpha library will resolve and download all your dependencies)#2018-06-0611:28alexmillerOn src/test paths - deps.edn doesn’t make this distinction. Normally I’d expects src to be part of the default :paths and test to be provided via an :extra-paths in an alias #2018-06-0611:29alexmillerOn tda version, I could probably add that version to verbose or describe#2018-06-0611:30cfleming@alexmiller Right, the issue is that I need t.d.a to get that info because I really have to parse deps.edn to get that level of detail.#2018-06-0611:30cflemingThat would be great.#2018-06-0611:31alexmillerOn source artifacts, nothing in clj, but you can easily use stuff in tda directly to do that. Surely IntelliJ already has something though? I guess maybe it relies on a pom?#2018-06-0611:31cflemingRight, that’s all in the Maven plugin which is an optional dependency.#2018-06-0611:32cflemingI’ve had problems with users disabling that and then wondering why e.g. autocomplete in project.clj doesn’t work.#2018-06-0611:33cfleming@alexmiller Is the deps alias going to stick around?#2018-06-0611:34cflemingIf so then I might be able to use clojure -A:deps -Spath, although it’s a little hacky.#2018-06-0611:36alexmillerNot planning on removing it but I hate to have you rely on that#2018-06-0611:37cflemingHaving some canonical way to build a classpath with the installed version of tda would be nice.#2018-06-0611:37alexmillerYeah#2018-06-0611:37cflemingEven putting the version in -Sdescribe punts to the client and forces it to support Maven resolution.#2018-06-0611:38cflemingHaving the classpath just include the uberjar would be fine too.#2018-06-0611:39alexmillerThe uberjar makes me feel dirty :)#2018-06-0611:39cflemingActually, perhaps just add the version and the uberjar path to -Sdescribe? Seems a bit internal… yeah#2018-06-0611:40cflemingBut, you know, it’s already there…#2018-06-0611:41alexmillerGive me a day or two to think about it#2018-06-0611:41cflemingSure#2018-06-0611:42cfleming@alexmiller Another feature request for the API would be to provide a download progress callback.#2018-06-0611:43cflemingThe Maven stuff supports that, it’s probably trickier for git.#2018-06-0611:43cflemingDo you use JGit or shell out?#2018-06-0611:43cflemingJGit might provide something.#2018-06-0611:44alexmillerJgit atm. They do have some callback stuff.#2018-06-0611:44cflemingIt would be great to expose that - downloading the deps of something like immutant can take forever.#2018-06-0611:45alexmillerExpose like via printing or by letting a caller get notification ?#2018-06-0611:46alexmillerWould be great to get tickets for this kinda stuff#2018-06-0611:47cflemingI can file some tomorrow.#2018-06-0611:48cflemingBy passing in a function which gets called back.#2018-06-0611:48cflemingThe pomegranate version looks like this: https://github.com/cemerick/pomegranate/blob/27b4ff3418bd989a52a6b0bd9df16c4954a92d8f/src/main/clojure/cemerick/pomegranate/aether.clj#L69-L110#2018-06-0611:49cflemingThat works nicely.#2018-06-0611:50cflemingAnd this:#2018-06-0611:50cfleminghttps://github.com/cemerick/pomegranate/blob/27b4ff3418bd989a52a6b0bd9df16c4954a92d8f/src/main/clojure/cemerick/pomegranate/aether.clj#L126-L137#2018-06-0611:51cflemingRight, bedtime. I’ll file some tickets tomorrow.#2018-06-0616:01Hugh PowellHi folks,
I'm working on a project to get leiningen to source deps.edn for dependency information (lein-tools-deps).
We currently shell out to clojure -Sdescribe to get the vector of :config-files.
This command also returns other information which requires resolution of all a projects dependencies (specifically :resolve-aliases, I think).
Is there any way to get just the :config-files (or all the information that doesn't need full resolution of all dependencies to be completed)?#2018-06-0616:49seancorfield@hugh336 Do you mean https://github.com/RickMoynihan/lein-tools-deps ?#2018-06-0708:03Hugh PowellYep, that's the one#2018-06-0715:09rickmoynihan@hugh336: Yeah it would be really useful if there was a variant on -Sdescribe that didn’t attempt to resolve your dependencies etc, and only returned the information that is known locally about the installation.
As for us in lein-tools-deps this means that we end up resolving aliases twice once when we don’t really want to#2018-06-0715:10rickmoynihanI don’t think there’s a good way around this without changing tools deps / or the clj/clojure command line project.#2018-06-0715:14alexmillerthere is actually a flag buried in the underlying clojure program to tell it not to resolve#2018-06-0715:15alexmillerbut it’s not surfaced out to the top#2018-06-0715:15alexmillerI’d be happy to have a ticket about the problem in TDEPS so I can think about a solution#2018-06-0715:16rickmoynihanwhat is the flag?#2018-06-0715:17alexmillerhttps://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/script/make_classpath.clj#L30#2018-06-0715:18alexmillerif set, that avoids doing the resolution, making a classpath, and saving those off#2018-06-0715:20alexmillerwas added for some stuff Stu was doing (in the guts of ion impl) where similar need arose#2018-06-0715:21rickmoynihanInteresting… I thought the issue was here as we shell out to run -Sdescribe
https://github.com/clojure/brew-install/blob/1.9.0/src/main/resources/clojure#L320-L332
though I could be misunderstanding the flow#2018-06-0715:22alexmilleryes, this flag is at a lower level and not exposed at the script level as I mentioned above#2018-06-0715:23rickmoynihancool… we can look at registering an issue#2018-06-0715:24alexmillerwait, -Sdescribe doesn’t resolve anything#2018-06-0715:24alexmillerso I will go all the way back to the top of this discussion and disagree with the premise#2018-06-0715:25rickmoynihanlol ok 🙂#2018-06-0715:25alexmillerso this: “Yeah it would be really useful if there was a variant on -Sdescribe that didn’t attempt to resolve your dependencies etc, and only returned the information that is known locally about the installation.” is not correct#2018-06-0715:26alexmillerthat’s exactly what -Sdescribe does#2018-06-0715:27rickmoynihanok it’s not what we’re seeing… maybe there’s something else causing us to misdiagnose the problem though#2018-06-0715:27alexmillerwell maybe there’s a bug there#2018-06-0715:27alexmillerlooking at the script, could definitely be but it’s just a matter of another flag check before resolving#2018-06-0715:28alexmillerso, I’d say that’s the ticket to file - ensure -Sdescribe doesn’t trigger resolution#2018-06-0715:30Hugh Powell@rickmoynihan @alexmiller Thanks for the discussion, I'll file a ticket#2018-06-0715:30alexmillersorry for my densitude#2018-06-0715:31rickmoynihannot at all, you must be pulled every which way#2018-06-0715:35rickmoynihan@alexmiller One other thing it looks like when using tools.deps as an API we don’t get the caching behaviour because it’s trapped inside the shell script here:
https://github.com/clojure/brew-install/blob/e706dbedbe91446ae0a09ff32fd36620cc966c55/src/main/resources/clojure#L238
I understand it’s probably there because it avoids you having to start another VM, but it seems to mean we don’t get the libs caching in our plugin. Is there a clj implementation of this somewhere, or do we need to write one?#2018-06-0715:56rcustodioHey… I’m using this org.imgscalr/imgscalr-lib {:local/root "libs/imgscalr-lib-4.3.jar"}
But when I do uber with boot, using seancorfield/boot-tools-deps, this jar is not extracted and added to the standalone jar, how do I do it, this is my boot command
BOOT_CLOJURE_VERSION=1.9.0 boot -d seancorfield/boot-tools-deps:0.4.5 \
boot-tools-deps.core/deps -B \
pom -p "zzk/catalog-image-svc" -v "0.0.1" -d "ZZK image processing" -u "" -l "X:X" \
uber \
jar -m clojure.main \
target
#2018-06-0716:16Hugh Powell@rickmoynihan @alexmiller Ticket created https://dev.clojure.org/jira/browse/TDEPS-82#2018-06-0716:18alexmillerthx#2018-06-0716:36seancorfield@rcustodio Does it work (run) if you use just clj, not boot? Can you build a JAR with depstar or pack?#2018-06-0716:36rcustodioYes, it works with run#2018-06-0716:36rcustodioclj run#2018-06-0716:37rcustodiodepstar pack, i dunno what that is#2018-06-0716:37seancorfieldCan you put the project up on GitHub so I can clone it and try it out?#2018-06-0716:37seancorfielddepstar and pack are clj-based utilities to create uberjars.#2018-06-0716:38rcustodiohmm.. I can’t because its a in-company project#2018-06-0716:38rcustodioThat is my deps
{:aliases
{:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.5.435"}}},
:test {:extra-paths ["test"]}},
:deps
{br.com.vikingmakt/augustus {:mvn/version "0.1.12"},
br.com.vikingmakt/njord {:mvn/version "0.3.0"},
cheshire {:mvn/version "5.8.0"},
clj-time {:mvn/version "0.14.4"},
com.novemberain/langohr {:mvn/version "5.0.0"},
com.novemberain/monger {:mvn/version "3.1.0"},
net.mikera/imagez {:mvn/version "0.12.0"
:exclusions [org.imgscalr/imgscalr-lib]},
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.0"},
org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.11.0"},
org.clojure/clojure {:mvn/version "1.9.0"},
org.clojure/tools.logging {:mvn/version "0.4.1"},
org.imgscalr/imgscalr-lib {:local/root "libs/imgscalr-lib-4.3.jar"}},
:mvn/repos
{"central" {:url ""}, "clojars" {:url ""}},
:paths ["resources" "src/main/clojure"]}
#2018-06-0716:39seancorfieldAnd that imgscalr-lib is your own custom in-house version?#2018-06-0716:39rcustodionope, I just got the last (master) from the github repository#2018-06-0716:39rcustodioBecause I’m using something that is not in the latest version from http://maven.org#2018-06-0716:42rcustodioI will try it out with depster#2018-06-0716:42seancorfield@rcustodio can you add -vv after -B on the deps task and send me the output via DM here? No point in clogging up the channel with it.#2018-06-0716:43rcustodiookey#2018-06-0716:43seancorfieldYou can find links to Depstar, Pack, and other tools.deps tooling here: https://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-06-0719:34kennyNot sure if this is a tools-deps problem but if I don't have AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY set as env vars, I get a access denied from a Maven repo. I am specifically trying to access the Datomic cloud repo "" which, I assume, does not require authenticated access.#2018-06-0721:34alexmillerYeah, same is under discussion in #datomic - I’m going to dig into it a little more#2018-06-0803:51cflemingAm I correct in thinking that aliases are not transitive? So if I have a git dep, there’s no way for me to use test code from it?#2018-06-0803:56cflemingIn fact, the same also seems to be true of local deps - is this intentional, or am I missing a way to work around this?#2018-06-0803:59cflemingI’m also confused by the fact that local deps don’t seem to be transitive. I have a test project, and it has a subproject which the root one depends on using :local/root. The subproject has a git dep, but that dep doesn’t appear on the classpath for the root project.#2018-06-0804:35ghadialiases being non transitive is intentional#2018-06-0804:36ghadiAs for the other comment, local deps should be transitive (provided a pom.xml or deps.edn)#2018-06-0902:51cfleming@ghadi Local deps are definitely not transitive at the moment - I haven’t tested git deps.#2018-06-0902:51ghadiBug#2018-06-0903:02cflemingActually, I might be going nuts - I can’t reproduce what I was seeing yesterday.#2018-06-0903:04cflemingI wonder if this was a caching problem:#2018-06-0903:05cfleming~/d/t/test-project (master)> tree .
.
├── deps.edn
└── sub-project
├── deps.edn
└── sub-sub-project
└── deps.edn
2 directories, 3 files
~/d/t/test-project (master)> cat deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "13e9d244678be7b235bb24a10310f9d147ea088d"}
test/sub-project {:local/root "sub-project"}}}
~/d/t/test-project (master)> cat sub-project/deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "0f06ca5eb08726e74b2dab8654923894b7ace6af"}
test/sub-sub-project {:local/root "sub-sub-project"}}}
~/d/t/test-project (master)> cat sub-project/sub-sub-project/deps.edn
{:deps {}}
~/d/t/test-project (master)> clojure -Spath
Error building classpath. Manifest type not detected when finding deps for test/sub-sub-project in coordinate #:local{:root "sub-sub-project"}
~/d/t/test-project (master)> clojure -Spath -C:test
test:src:/Users/colin/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/Users/colin/.gitlibs/libs/org.clojure/data.json/13e9d244678be7b235bb24a10310f9d147ea088d/src/main/clojure:/Users/colin/dev/tools.deps.alpha/test-project/sub-project/src:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar
#2018-06-0903:05cflemingHere the first classpath with no alias gives an error that the manifest is not supplied.#2018-06-0903:06cflemingThe second doesn’t give that error, but returns an incorrect classpath.#2018-06-0903:06cflemingI add the manifests:#2018-06-0903:06cfleming~/d/t/test-project (master)> tree .
.
├── deps.edn
└── sub-project
├── deps.edn
└── sub-sub-project
└── deps.edn
2 directories, 3 files
~/d/t/test-project (master)> cat deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "13e9d244678be7b235bb24a10310f9d147ea088d"}
test/sub-project {:local/root "sub-project"
:deps/manifest :deps}}}
~/d/t/test-project (master)> cat sub-project/deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "0f06ca5eb08726e74b2dab8654923894b7ace6af"}
test/sub-sub-project {:local/root "sub-sub-project"
:deps/manifest :deps}}}
~/d/t/test-project (master)> cat sub-project/sub-sub-project/deps.edn
{:deps {}}
#2018-06-0903:07cflemingAnd then both work:#2018-06-0903:07cfleming~/d/t/test-project (master)> clojure -Spath
src:/Users/colin/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/Users/colin/.gitlibs/libs/org.clojure/data.json/13e9d244678be7b235bb24a10310f9d147ea088d/src/main/clojure:/Users/colin/dev/tools.deps.alpha/test-project/sub-project/src:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar:/Users/colin/dev/tools.deps.alpha/test-project/sub-sub-project/src
~/d/t/test-project (master)> clojure -Spath -C:test
test:src:/Users/colin/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/Users/colin/.gitlibs/libs/org.clojure/data.json/13e9d244678be7b235bb24a10310f9d147ea088d/src/main/clojure:/Users/colin/dev/tools.deps.alpha/test-project/sub-project/src:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar:/Users/colin/dev/tools.deps.alpha/test-project/sub-sub-project/src
#2018-06-0903:08cflemingThen if I remove the manifest, I get the error with both:#2018-06-0903:08cfleming~/d/t/test-project (master)> cat deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "13e9d244678be7b235bb24a10310f9d147ea088d"}
test/sub-project {:local/root "sub-project"}}}
~/d/t/test-project (master)> cat sub-project/deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "0f06ca5eb08726e74b2dab8654923894b7ace6af"}
test/sub-sub-project {:local/root "sub-sub-project"}}}
~/d/t/test-project (master)> clojure -Spath
Error building classpath. Manifest type not detected when finding deps for test/sub-sub-project in coordinate #:local{:root "sub-sub-project"}
~/d/t/test-project (master)> clojure -Spath -C:test
Error building classpath. Manifest type not detected when finding deps for test/sub-sub-project in coordinate #:local{:root "sub-sub-project"}
#2018-06-0904:08alexmillertoo tired to read all that atm but there is a known issue that the caching will not notice when local dep project classpaths change, so you need to -Sforce to recompute. not sure if that’s what you’re seeing. I have a plan to fix that just haven’t had time to work on it yet.#2018-06-0904:09alexmillerif you think there’s a bug, please feel free to drop it in a ticket so I can look at it later as this will get lost#2018-06-0919:49gfrederickswhat's the right way to call clojure with a script going to stdin?
it works okay with no args, but prints a bunch of repl prompts I don't want#2018-06-0921:24alexmillerWe don’t have that level of repl control with clojure.main#2018-06-0921:25alexmillerWell, maybe the repls not involved there?#2018-06-0921:27gfredericks$ echo '(println 42)' | clojure
Clojure 1.9.0
user=> 42
nil
user=>
is what I'm referring to#2018-06-0921:28alexmillerIsn’t there a - option you can pass?#2018-06-0921:29gfredericksthere is, that's probably all I need#2018-06-0921:42richiardiandreaOh awesome didn't know that either 😃#2018-06-1015:39borkdudeCool, I needed this once too#2018-06-1016:07ghadiI do that all the time. I like datomic ion's take on it too.#2018-06-1016:12ghadiSending in an edn map through an arg or stdin#2018-06-1017:00gfredericksironically I quickly realized that I couldn't do that in this case, because the program also took its input from stdin#2018-06-1017:01gfredericksI guess in theory I could have arranged for the script to be prepended to the input and make sure the last form is one that does the work and then System/exits, but that sounds pretty hacky#2018-06-1017:05borkdude@gfredericks I once asked for an option in here to provide a program with the -e option, without printing expressions.#2018-06-1017:05borkdudeWould that make sense in your case?#2018-06-1017:05gfredericksI could've but it was a ~20 line script so I felt weird putting that in an arg#2018-06-1017:06borkdudeput it in a variable.#2018-06-1017:06dominicmclojure -e <<EOF
(println "ta-da")
EOF
#2018-06-1017:06dominicmheredoc can be pretty useful#2018-06-1017:07dominicm(apparently that doesn't work)#2018-06-1017:07borkdude{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "f0b29f829b94859495b0929f829b94859495"}, :content ("[email protected]")}#2018-06-1017:08borkdudeBut then again, you could just do clj hello.clj 🙂#2018-06-1017:13gfredericksI'm not a bash lawyer yet, but I got the distinct impression yesterday that <<EOF goes to stdin#2018-06-1017:23dominicmclojure -e "$(cat << 'FOO'
(println "ta-da")
FOO
)"
You're right, I've just got on a journey learning that 🙂#2018-06-1017:24gfredericksI've been learning some bash recently.
Weirdest thing so far is the <> redirection operator.#2018-06-1017:26dominicmwhat does that do?#2018-06-1020:08gfrederickssame as < except it makes stdin writable#2018-06-1112:02delaguardoIs it possible to specify init-opt within deps.edn file?#2018-06-1112:19alexmillerNo, not right now#2018-06-1117:11bhaumanshould I be surprised that clojure forms in main opts are not output as quoted args?#2018-06-1117:11bhaumanI.E. :main-opts ["-e" (list 1 2 3)]#2018-06-1117:12bhaumanthe list expression gets broken up into separate args#2018-06-1117:13bhaumanI guess escaping gets complicated#2018-06-1117:14bhaumanor there is a depth of features here I don't understand#2018-06-1117:21hiredmanI think the thing you might be missing is :main-opts is meant to be a list of strings passed as command line arguments, not a list of arbitrary datastructures#2018-06-1117:25seancorfieldAlso remember that , is whitespace so you can often get around shell-level tokenization by using , in strings instead of spaces. But, yeah, :main-opts are strings @bhauman.#2018-06-1117:26seancorfieldSee https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L33-L37 for example...#2018-06-1117:26bhaumanthanks guys#2018-06-1117:27bhaumananother question, is this not a feature that we might want?#2018-06-1117:28bhaumanto take advantage of the fact that we are in a clojure form to begin with and script our way to euphoria?#2018-06-1117:29seancorfieldBut :main-opts are what is passed as a sequence of strings to the -main function via clojure.main -- it's all strings. You can easily pass an EDN string if you want a data structure.#2018-06-1117:29seancorfieldThere's no difference between command-line invocation and :main-opts.#2018-06-1117:30seancorfieldand it is passed by a script, not by Clojure code.#2018-06-1117:31bhaumanthe script doesn't parse the deps.edn#2018-06-1117:31seancorfieldThe :main-opts are cached to a text file and then piped into the (string) options passed to clojure.main. Just like the :jvm-opts.#2018-06-1117:31bhaumandoes it?#2018-06-1117:31bhaumanI get this but I'm sure clojure code prepares the opts#2018-06-1117:31bhaumanbefore caching#2018-06-1117:33seancorfieldIt's just strings in the cached file. And it's just strings that are passed to clojure.main.#2018-06-1117:34bhaumanI must be missing something, are you saying that the feature isn't possible?#2018-06-1117:35bhaumanor that it messes up the semantics?#2018-06-1117:35seancorfieldTake a look at the clojure shell script. Here's an example of what gets cached: (! 763)-> ls -l .cpcache/*.main
-rw-r--r-- 1 sean staff 90 Mar 26 15:38 .cpcache/1770942858.main
-rw-r--r-- 1 sean staff 17 Apr 14 00:52 .cpcache/2142975460.main
-rw-r--r-- 1 sean staff 17 Apr 14 18:18 .cpcache/4269137912.main
-rw-r--r-- 1 sean staff 22 May 2 14:38 .cpcache/84686696.main
(sean)-(jobs:0)-(~/clojure)
(! 764)-> cat .cpcache/84686696.main
-m rebel-readline.main
(sean)-(jobs:0)-(~/clojure)
(! 765)-> cat .cpcache/1770942858.main
-e (require,'[clojure.tools.nrepl.server,:refer,[start-server]]),(start-server,:port,5555)
(sean)-(jobs:0)-(~/clojure)
(! 766)->
#2018-06-1117:37bhaumanyes but we call the tools.aplha to resolve the tags#2018-06-1117:38bhaumanwhich is parsing main opts and therefore could quote clojure forms before caching them?#2018-06-1119:07hagmonkI just made a fairly big update to https://github.com/hagmonk/depify/ … would appreciate any folks here who could kick the tires a bit before I post to the mailing list#2018-06-1119:14seancorfieldThis includes adding some test runner boilerplate from clj-new. -- not sure what you mean there @hagmonk?#2018-06-1119:15seancorfieldShould that link to Cognitect's test-runner repo perhaps?#2018-06-1119:17hagmonkOh I pinched that from a clj-new generated app - specifically the separate test and runner aliases #2018-06-1119:18hagmonkI didn’t see the same style being used on the test-runner repo, but maybe I missed it #2018-06-1119:19seancorfieldAh, I see. Linking to my dot-clojure repo probably makes more sense then for that https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L9-L19 (at least by way of explanation).#2018-06-1119:19hagmonkCool! I will make that change#2018-06-1119:22hagmonkI’m exploring the idea of adding more aliases to provide parity with project.clj, in those cases where someone has a tools.deps based tool we can use #2018-06-1119:22hagmonkCLJS and uberjars for instance #2018-06-1119:25hagmonkso now I'm looking at your deps.edn file and getting lots of ideas :) but I need to work on my real projects today at some point, LOL#2018-06-1120:21alexmiller@bhauman I didn’t understand your earlier comments re quoting. if it’s something I should pay attention to, please try again :)#2018-06-1120:21bhaumanhappy to 🙂#2018-06-1120:21alexmillerthe ground truth is that the jvm takes a classpath, a main class and string args to the main class. so we want to expose that, first.#2018-06-1120:22alexmillerother things on top of that may also be useful#2018-06-1120:22bhaumanabsolutely#2018-06-1120:22alexmillerbut I didn’t understand what problem you were trying to solve or thing you were trying to make better or avoid#2018-06-1120:23bhaumanit just seems there is an opportunity when providing clojure forms (not strings) inline in :main-opts to output a single string arg that will make it through to the jvm process as a single arg#2018-06-1120:25alexmillerthe ultimate call needs individual string args though - how do you decide how to parse and tokenize?#2018-06-1120:28bhauman:main-opts ["-e" {:build 1}] => clojure -e "{:build 1}"#2018-06-1120:28bhaumanis what I'm imagining#2018-06-1120:29alexmilleryou seem to be suggesting a read/print cycle but using different kinds of printing#2018-06-1120:30alexmiller“-e” would be read as a string and pr’ed as “-e” (perhaps escaped in the invocation as clojure -e '"-e"`#2018-06-1120:32bhaumanit seems that string typs can go though untouched and that forms would be dispatched as such#2018-06-1120:32bhaumanother than string forms 🙂#2018-06-1120:32bhaumanits a common need, and it seems like a shame to be in a clojure form and then have to translate and escape a clojrue form correctly#2018-06-1120:33alexmillerthere are a couple layers of transformation here and I intend to sit down and sort through them methodically. I’m well aware it’s broken in various ways at the moment.#2018-06-1120:33alexmillerI’m a little concerned about implying evaluation in the deps.edn file#2018-06-1120:34alexmillerthe args there are read (as edn) and never evaluated and I don’t want to imply that (or do it)#2018-06-1120:34alexmillerit’s important to me that it remains plain edn data#2018-06-1120:35bhaumanabsolutely#2018-06-1120:36bhaumannot what I'm thinking or wanting to imply at all#2018-06-1120:36alexmillerI didn’t think so, but it’s easy to get that impression from that example#2018-06-1120:36alexmillerI’m not a fan of “special” arg handling#2018-06-1120:36bhaumanwhere I really want it is for cljs.main and figwheel.main to supply inline config#2018-06-1120:37alexmilleryeah, that’s really same need as say, server repl#2018-06-1120:37alexmillerpassing arbitrary edn as an arg#2018-06-1120:38bhauman:main-opts ["-m" "cljs.main" "-co" {:output-to "out/main.js"} ]#2018-06-1120:38bhaumanyeah#2018-06-1120:38alexmillerin a way that is not a nightmare, and that works :)#2018-06-1120:38bhaumanwhat works?#2018-06-1120:38alexmillerI’m just saying that a requirement is that the end result works#2018-06-1120:38alexmiller(as it’s broken now)#2018-06-1120:39bhaumangotcha 🙂#2018-06-1120:40bhaumanit is interesting though that I'd expected this to work#2018-06-1120:40bhaumanI intuited that it might me a likely feature#2018-06-1120:40alexmillerthere might be a higher level solution to this problem too - I know Rich has mentioned various ideas that are in this ballpark but a somewhat different take on it#2018-06-1120:41alexmillerfor example, you could expose functions directly and hide the main goo entirely#2018-06-1120:42bhaumanthat is interesting#2018-06-1120:42alexmillerit’s come up in several contexts and I know he’s noodling about it#2018-06-1120:42alexmillerbut sometimes he noodles for many years before it lands somewhere :)#2018-06-1120:43alexmillerso I make no suggestion that a deux ex machina will save us all here#2018-06-1120:44alexmillerclojure.main itself is both very flexible and frustratingly limited#2018-06-1120:44alexmillerlike I think it sucks that rebel-readline couldn’t just be hooked directly#2018-06-1120:44bhaumanI had some thoughts about that#2018-06-1120:45bhaumanI could potentially graalvm it, and provide a rlwrap equivalent#2018-06-1120:45alexmillerwell, I need to roll back to other work, but I would like to hear them somewhere :)#2018-06-1120:46bhaumanthanks Alex! later#2018-06-1120:46alexmillerlater…#2018-06-1210:54borkdudehmm, native rebel-readline… interesting#2018-06-1220:58hagmonkIs there some escaping trick for supplying edn inside tools.deps aliases?#2018-06-1220:58hagmonk:cljs {:main-opts ["-m" "cljs.main" "--compile-opts" "{:asset-path \"out\" :output-dir \"resources/public/out\"}" "--compile" "access-manager.client" ]}}
#2018-06-1220:58mfikesUse a lot of non-whitespace whitespace (aka ,#2018-06-1220:59bhaumanreplace the whitespace with commas#2018-06-1223:50richiardiandreawouldn't go around telling colleagues about this though, we might be mistaken for a bunch of JS devs 😄 😄 🙂#2018-06-1223:56zentropeWith deps.edn + cljs.main + figwheel.main, the tooling is so simple and streamlined that I just can't see how JS devs put up with that they do.#2018-06-1300:01hagmonkwhere's the cutting edge example for figwheel.main? I was just about to go looking for that#2018-06-1300:02hagmonkI am thinking of adding it to https://github.com/hagmonk/depify, so that existing figwheel projects can be more easily converted over#2018-06-1300:02bhauman@hagmonk working on a template right now#2018-06-1300:03bhaumanfigwheel-main will be a tough conversion though#2018-06-1300:03bhaumanconfig names are different#2018-06-1300:04bhaumanbut actually it should be pretty straightforward for most projects#2018-06-1300:07hagmonkI'm sure there are some leiningen-isms that are not possible to bring across, or considered an anathema in the eyes of deps.edn, but I'd like to bootstrap people to the greatest extent possible#2018-06-1300:07hagmonkI'm already generating Makefiles for my internal projects, so that particular pandora's box is open#2018-06-1300:19bhaumanYeah just created a Makefile myself, I forgot how dang effective they are#2018-06-1300:19hagmonkRight?? Quick and painless … it had been at least 10 years since I wrote one#2018-06-1300:20bhaumanyeah and it can get pretty sophisticated with relatively little syntax#2018-06-1300:20bhaumanits like it was built for this particular job#2018-06-1300:21hagmonkJust remember that literal $ signs need to be $$ to stop make from interpreting them :) Got caught by that one#2018-06-1300:22bhaumanI have noticed something interesting, if you use lein without plugins and only use the profiles and aliases, the performance is pretty similar#2018-06-1300:22bhaumanor significantly improved#2018-06-1300:23hagmonkfascinating … perhaps plugin dependencies are in serial after everything else#2018-06-1300:23bhaumanthey are dynamicly loaded#2018-06-1300:24bhaumanwhich is what causes the hit#2018-06-1300:27hagmonkinherent to changing the classpath dynamically, or something specific with clojure like having to compile?#2018-06-1300:27bhaumanclojure needing to compile#2018-06-1300:30hagmonkIf the jar is in the classpath beforehand, is compilation skipped or just faster because the JVM gets to batch it all up at once?#2018-06-1300:32bhaumanlein is aot compiled#2018-06-1300:33bhaumanso... fast by itself#2018-06-1300:33zentropeIf someone made a super quick class-path-maveny thing in non-Java, make would sing.#2018-06-1301:17alexmillerthat would be very tedious#2018-06-1301:17alexmillerhowever, you could graalvm the classpath maker#2018-06-1302:59hagmonkhmm, @bhauman - java.lang.RuntimeException: No such var: cljs.repl/*repl-env*#2018-06-1303:00bhauman@hagmonk that would happen if you were somehow getting an earlier version of CLJS than 1.10#2018-06-1303:01bhaumanwe can talk about this in #figwheel-main#2018-06-1303:01hagmonkah, didn't know that existed :)#2018-06-1317:15bhaumantools-deps does a silent failure when an alias doesn't exist#2018-06-1317:17dominicms/silent failure/ignores#2018-06-1317:37alexmillerticket welcome#2018-06-1317:37alexmillerI thought I actually had a check for that, but maybe not#2018-06-1317:39dominicmI don't know it's strictly a problem tbh#2018-06-1317:47alexmillerseems like it’s more likely to be a typo than cleverness#2018-06-1319:10hagmonk13th law of software development: itches are to be scratched by the one who itches.#2018-06-1319:10alexmillerI’ve used the lein stuff for this a bit and it’s SUPER slow#2018-06-1319:11alexmillerso slow that I just go use the equivalent web pages instead#2018-06-1319:11alexmillerI have no idea why#2018-06-1319:11alexmillerbut maybe worth understanding#2018-06-1319:12alexmillersome next level stuff would be to think about bridging the gap between ns and dep#2018-06-1319:12dominicm@alexmiller what are the important features of the search? I've got a week of open source and JUXT are interested in deps.edn.#2018-06-1319:13alexmillerlein search does approximately what I want, I just find it too slow to use#2018-06-1319:13alexmillerI want google for deps#2018-06-1319:13dominicmStartup or runtime? It will almost certainly be called find, because 4 letters.#2018-06-1319:13alexmillerruntime is the part I’m referring to#2018-06-1319:14dominicmBut mavens is fast?#2018-06-1319:14alexmillerI think there might be some giant index file that gets downloaded periodically?#2018-06-1319:14alexmillerI don’t remember what maven has for this#2018-06-1319:14alexmillermaybe I’m wrong that they do#2018-06-1319:15alexmillerI just tried lein search again and it was pretty fast#2018-06-1319:15alexmillerbeen a while#2018-06-1319:16alexmillerhow about this - I’d like to do lein search functionality without being tied to lein#2018-06-1319:16dominicmOr tools.deps.alpha!#2018-06-1319:16dominicmJust a main#2018-06-1319:17alexmillerno reason it needs to be there#2018-06-1319:18dominicmI agree.#2018-06-1319:18alexmillerI don’t think maven actually has apis for this - it’s site-specific functionality iirc#2018-06-1319:18alexmiller(which also means they are different :)#2018-06-1319:18dominicmClojars has an index which mentions nexus.#2018-06-1319:19alexmillerwell, maybe they’re the same then, not sure#2018-06-1319:19dominicmMaybe. No idea. Just a data point.#2018-06-1319:19alexmillerthe next level and far more interesting problem is how to connect namespaces and artifacts#2018-06-1319:20dominicmI'm super interested in this.#2018-06-1319:20dominicmSolving this makes writing plugins easier.#2018-06-1319:20dominicmBoth for different kinds of artefact, and for build tools.#2018-06-1319:20alexmillerRich and I have talked about it a lot but it’s a place where Maven doesn’t really help you much#2018-06-1319:23dominicmWhat would a straw man solution look like? I'm struggling to picture something.#2018-06-1319:30alexmillerI don’t have one :)#2018-06-1319:30alexmillersorry, other things I need to focus on atm#2018-06-1322:32danielcomptonLein now uses the Clojars api iirc. It used to download some kind of massive index which was very slow#2018-06-1400:03alexmillerGlad I’m not imagining that, and that it doesn’t now! It did seem plenty fast#2018-06-1323:20bhauman@seancorfield trying to use clj-new with a locally installed lein template and not getting very far#2018-06-1323:32seancorfield@bhauman Can you be a bit more specific?#2018-06-1323:32seancorfield"locally installed"?#2018-06-1323:33bhaumanI did a lein install on a template I'm working on locally#2018-06-1323:34bhaumanshould it be able to resolve it?#2018-06-1323:34bhaumanas it dies in clj_new.helpers$resolve_remote_template.invokeStatic(helpers.clj:64)#2018-06-1323:34seancorfieldHmm, my normal process for testing templates is to install them locally and then try to use them via new...#2018-06-1323:35bhaumanhere is the message#2018-06-1323:35bhaumanException in thread "main" clojure.lang.ExceptionInfo: Unable to locate Clojure's edn files {:exit 1, :out "Invalid option: -Sdescribe\n", :err ""}#2018-06-1323:35seancorfieldAh, that's where it shells out (via tools.deps.alpha) to run clojure -Sdescribe...#2018-06-1323:36seancorfieldSo the error is coming from tools.deps.alpha not being able to run that command.#2018-06-1323:36bhaumanand so there needs to be a deps.edn in the current dir?#2018-06-1323:36seancorfieldIf you type clojure -Sdescribe in the folder where you are running clj-new what do you get?#2018-06-1323:36seancorfieldNo, this isn't about deps.edn#2018-06-1323:37bhaumaninvalid option#2018-06-1323:37bhaumanI have to update#2018-06-1323:37seancorfieldHow old is your version of clojure?#2018-06-1323:37seancorfieldHere's what I see in a folder without deps.edn: (! 665)-> clojure -Sdescribe
{:version "1.9.0.375"
:config-files ["/usr/local/Cellar/clojure/1.9.0.375/deps.edn" "/Users/sean/.clojure/deps.edn" ]
:install-dir "/usr/local/Cellar/clojure/1.9.0.375"
:config-dir "/Users/sean/.clojure"
:cache-dir "/Users/sean/.clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
#2018-06-1323:38seancorfieldI thought -Sdescribe had been in there for ages... 👀#2018-06-1323:38bhaumanor not#2018-06-1323:39bhaumanyeah its not in my help string#2018-06-1323:40bhaumanso upgrading#2018-06-1323:41bhauman@seancorfield and you are enforcing that project names are multisegment and qualified?#2018-06-1323:41bhaumanshouldn't that be up to the template?#2018-06-1323:42seancorfieldBy popular demand, I am disallowing single-segment project names, yes. You can either use a multi-segment name or a qualified name (or both).#2018-06-1323:42seancorfieldSome templates don't accept a qualified name -- which I consider a bug in those templates 🙂#2018-06-1323:42bhauman@seancorfield it worked 🙂 but I'd like to be able to specify a single segement name like lein allows#2018-06-1323:43seancorfieldIf they don't accept a qualified name, they should accept a multi-segment name.#2018-06-1323:43bhaumanthe lein app template morphs a single segment name into a name.core#2018-06-1323:44seancorfieldYes, which is a terrible idea.#2018-06-1323:44seancorfieldEven Phil says so.#2018-06-1323:44seancorfieldYou can always do this: (! 666)-> clj -A:new app bhauman.core
Generating a project called bhauman.core based on the 'app' template.
Wed Jun 13 16:43:52
(sean)-(jobs:0)-(~/clojure/scratch)
(! 667)-> tree bhauman.core/
|____
| |____.gitignore
| |____.hgignore
| |____CHANGELOG.md
| |____deps.edn
| |____doc
| | |____intro.md
| |____LICENSE
| |____README.md
| |____resources
| | |____.keep
| |____src
| | |____bhauman
| | | |____core.clj
| |____test
| | |____bhauman
| | | |____core_test.clj
#2018-06-1323:46gfredericksis (! 666)-> your bash prompt#2018-06-1323:47gfrederickser; I guess the last of three lines of it#2018-06-1323:47seancorfieldAh, yes, my crazy bash prompt... just a sec, I'll get the code for that...#2018-06-1323:48seancorfield#
PS1="\n\d \t\n\[\e[32;1m\](\[\e[37;1m\]\u\[\e[32;1m\])-(\[\e[37;1m\]jobs:\j\[\e[32;1m\])-(\[\e[37;1m\]\w\[\e[32;1m\])\n(\[\[\e[37;1m\]! \!\[\e[32;1m\])-> \[\e[0m\]"
#2018-06-1323:48bhauman@seancorfield OK well I don't see the disturbing problem with using a single segment when you are developing a webapp#2018-06-1323:48bhaumanthats going to be deployed to heroky#2018-06-1323:48bhaumanI should call it heroky from now on#2018-06-1323:49seancorfieldThen use <whatever>.core as your project name and you'll get what Leiningen does.#2018-06-1323:49bhaumanthanks I understand that, was just wanting to keep things familiar for folks who are used to doing lein new figwheel hello-world#2018-06-1323:50bhaumanbut not a big deal#2018-06-1323:50bhaumanclj -Anew figwheel-main hello.core works#2018-06-1323:53seancorfieldhttps://github.com/bbatsov/clojure-style-guide/pull/100#2018-06-1323:53seancorfieldSee comments about ClojureScript in that thread, as well as Alex's comment.#2018-06-1323:54bhaumanyou do realize I'm not advocating single segment namespaces?#2018-06-1323:54seancorfieldYou seemed to be when you complained that clj-new disallowed them 🙂#2018-06-1323:55bhaumana single segment argument atumatically gets a "core" segment appended#2018-06-1323:55seancorfieldI haven't -- yet -- disallowed them in boot-new but I may well propose that.#2018-06-1323:56bhaumansingle segments don't work I get that#2018-06-1323:56seancorfieldYes, and having thousands of projects with spurious .core namespaces, just because we were all too lazy to teach beginners to name things properly, is not a great situation to be in.#2018-06-1323:59bhaumanOK so I'm not asking for single-segment namespaces, and defaulting to core is a bad idea for an initial template#2018-06-1400:00bhaumanI'm also betting that defaulting to "app" rather than core doesn't work for you either#2018-06-1400:02seancorfieldPeople should be explicit about their project namespace.#2018-06-1400:05bhaumanand my point of view is that the name in the template can be more general than a project name, and it can be helpful to let folks who write templates do what they want.#2018-06-1400:06bhaumanthe clj generators come in handy there#2018-06-1403:11hagmonkJust a heads up, I pushed a new https://github.com/hagmonk/depify that now includes the ability to read project.clj from stdin, as well as interpreting output from lein pprint#2018-06-1420:57hagmonkHey so riddle me this ... has any thought been given to using deps.edn in the context of a monorepo? #2018-06-1420:58hagmonkI’m not sure it’s currently possible to supply a path to a directory inside a larger git repo - perhaps I just missed that feature? #2018-06-1421:04seancorfieldWe're working with a monorepo at work and slowly figuring that out. Not a priority tho' so progress is a bit slow. #2018-06-1421:06alexmilleras a git dep or a local dep?#2018-06-1421:07alexmillerthere are some ways with :deps/root etc to do some of this but also some known issues#2018-06-1421:10seancorfieldI'm working with local deps so far. #2018-06-1421:14hagmonkI guess in the context of the monorepo itself, you would use :deps/root to point at other resources in the monorepo ... but clients outside the monorepo have no way to point at a subfolder’s deps.edn #2018-06-1421:15hagmonkA real world example would be figwheel-main’s deps.edn: https://github.com/bhauman/lein-figwheel/blob/master/figwheel-main/deps.edn#2018-06-1421:15hagmonkAFAICT, there’s no way I can depend on that deps.edn file, because it’s not at the root of the repo #2018-06-1421:28kenny@hagmonk Setting :deps/root to "figwheel-main" doesn't work?#2018-06-1421:28hagmonkOh, perhaps that is the thing I’m missing! #2018-06-1421:29kenny{:git/url ""
:sha "6368c0e49108fc7a8ec99df05831ff289123f1a5"
:deps/root "figwheel-main"}
🙂#2018-06-1421:29hagmonkThanks @kenny! Had no idea that existed#2018-06-1421:30kennyYeah I had the same problem. I don't think it's documented yet.#2018-06-1422:23alexmillerit’s not#2018-06-1422:24bhaumangood to know!#2018-06-1515:38richiardiandrea@michal interesting work you have shared there thank you!#2018-06-1515:46michal@richiardiandrea you're welcome. this is something I had on my mind since I saw JUXT's edge project first time. I hope it will be useful at least for some people here 🙂#2018-06-1601:16hagmonk@richiardiandrea @michal what are you guys referring to? I think it might have scrolled out of the channel history …#2018-06-1601:24richiardiandrea@hagmonk this -> https://github.com/mbuczko/revolt#2018-06-1601:26hagmonkThat's cool!#2018-06-1601:27hagmonkhow did I not know about Capsule before this …#2018-06-1604:06dominicmIt took me so long to find#2018-06-1604:06dominicmAnd I was really looking#2018-06-1700:31hagmonkAlright @alexmiller, I've almost finished that "google for deps without lein" tool you were wanting :) Just putting the finishing touches on the readme …#2018-06-1701:08hagmonkAlright, would love if folks here could kick the tires on this and tell me what you think: https://github.com/hagmonk/find-deps#2018-06-1701:09hagmonkI think this is one tool I actually started using while I was in the middle of writing it. Didn't realize how badly I wanted it :) No more going to google to find dep coords#2018-06-1702:30mfikes^ Very nice 🙂#2018-06-1721:09hagmonkOk, who here knows something about stdin/stdout flushing? I'm adding the ability to pipe find-deps commands together, and having a devil of a time getting that to work#2018-06-1721:38hagmonkfor posterity:
1) shopt -u lastpipe is evil, and I had to disable that (`shopt -s lastpipe`)
2) (print "") (flush) at the top of -main will prime the stdout stream such that other processes connected via pipes can test for this condition with (pos? (.available System/in))#2018-06-1722:44hagmonkLatest find-deps was just pushed, it now has the ability to write updates to your deps.edn file. I posted an announcement to the Clojure list too#2018-06-1722:45hagmonkE.g. adding metosin/jsonista is now as easy as: clj -A:find-deps -F:save jsonista#2018-06-1722:45hagmonkI haven't really found any cases yet where the fuzzy matching has picked totally the wrong lib - that's probably the number one thing you can bring to my attention if you see it fail#2018-06-1801:41cfleming@hagmonk That looks lovely!#2018-06-1801:42cflemingSearching is by artifact/group name, right? AFAIK you can’t use the Maven APIs to search for namespace names?#2018-06-1801:43hagmonkyes, just the group/artifact string#2018-06-1801:44cflemingWe really need something like Maven’s search by classname for namespaces and vars, but the indexing required for that is non-trivial.#2018-06-1801:45hagmonkright, I did some poking around the crossclj sources, seems like the fellow maintaining that has taken a step back from it#2018-06-1801:45cflemingYes, he has#2018-06-1801:46cflemingI’m not sure he maintained it much past the initial implementation, but I might be wrong there.#2018-06-1801:46hagmonkIt would be great to have a CLJ/CLJS source code index that also tracked the source repo and any built artifacts spun out of that#2018-06-1801:47cflemingI have a question about deps support for Cursive. As I envision it, each deps project (i.e. a deps.edn, basically) would be a module in IntelliJ, so modules will be created for local deps etc, and libraries created for git deps.#2018-06-1801:48cflemingMy problem is that modules have to be given a name, and it has to be something that I can uniquely determine from the deps.edn file, which AFAIK doesn’t provide anything like that.#2018-06-1801:48cflemingThe only thing that I can think of is to use the name of the containing directory. Does that seem like a reasonable convention?#2018-06-1803:12alexmillerWe have talked about this a lot in the context of git deps. I don’t have time to write it all now will try to hit this tomorrow#2018-06-1801:49cflemingI could also allow a name to be provided in a :cursive/name my-module entry in the deps map, assuming that sort of thing is allowed.#2018-06-1801:50cflemingOh, and one restriction is that modules have to have unique names.#2018-06-1801:51hagmonkI think the general sentiment has been to keep deps.edn uncluttered with anything not related to building a classpath or running the program. Whether unrelated keys are preserved seems to be by chance rather than design#2018-06-1801:51cflemingOk#2018-06-1801:52hagmonkbut you just need a name for the module, it doesn't need to be the same as, say, the official maven artifact name#2018-06-1801:52cflemingHowever, I doubt Cursive is the only system that would find it useful to assign a name to a project.#2018-06-1801:52cflemingTrue, but there’s literally nothing there to base it off right now.#2018-06-1801:52cflemingI mean, I could use the sha of the path to the project, but no-one is likely to be happy with that 🙂#2018-06-1801:53hagmonkhaha, what? who here doesn't love a full length SHA? :)#2018-06-1801:54cflemingHey, it’s (almost) guaranteed unique!#2018-06-1801:55hagmonkI'm just looking through the tools on https://github.com/clojure/tools.deps.alpha/wiki/Tools (which I have to update with find-deps and mbuczko/revolt) to see what conventions exist for project name, in those cases where one is needed#2018-06-1801:56cflemingAs I understand it, if there are two projects which both refer to a common library using a local ref, there’s no requirement even for the name they assign to it to be unique.#2018-06-1801:56cflemingi.e. project1: foo/bar {:local/root "../lib"} and project2: baz/bup {:local/root "../lib"}#2018-06-1801:57cflemingIn fact, I have no idea where those names are used, if anywhere#2018-06-1801:58hagmonkI suppose the resolved path to the dep is what actually matters there, rather than the symbol. I'm not aware of any use for it either#2018-06-1801:58hagmonkI guess one could also look at any git metadata in the project folder - perhaps prefer that, then fall back to directory name#2018-06-1801:59cflemingI guess I could use the path from the main project root as the module name, but that will get weird if the local root is outside the main project.#2018-06-1801:59cflemingI don’t know if IntelliJ has restrictions on what modules can be called, actually.#2018-06-1802:00cflemingThe problem with git metadata is that Cursive then needs git capabilities as well as maven capabilities#2018-06-1802:00hagmonkplus it won't always exist#2018-06-1802:00cflemingRight.#2018-06-1802:01hagmonkperhaps the basename of the resolved path of the dep is good enough for most cases? If there is a name clash it might be worth foisting that on the user anyhow#2018-06-1802:01cflemingThat’s basically the name of the containing directory, right?#2018-06-1802:02hagmonkRight - or even "basename-1/basename" ... most of my projects would be sensibly named with that convention#2018-06-1802:02cflemingI guess I’m worried about cases like server/api/deps.edn and client/api/deps.edn#2018-06-1802:02cflemingYeah#2018-06-1802:02cflemingI’ll see if I can find out whether there are restrictions on module names.#2018-06-1802:03cflemingI think I’ll just have to implement things until people stop complaining.#2018-06-1802:04hagmonkThat is the zen of software development right there!#2018-06-1802:04cflemingThe problem with foisting it on the user is that there’s no way to store a customisation for the name within the IntelliJ project - I have to return a project structure based on a list of deps.edn files, no more.#2018-06-1802:04cflemingIndeed - I think of it as a dynamic search algorithm.#2018-06-1802:05cfleming@alexmiller I’m interested if there’s an official position on keys like :cursive/name my-module in deps files.#2018-06-1803:01seancorfield@cfleming I don't know how "official" it is but whenever the subject has come up, Alex has always discouraged people adding "random" keys to the file...#2018-06-1803:05hagmonkAlso anecdotal, but listening to Rich on the cognitect podcast (when will they fix the audio?) I heard him say how nice it was to look at a deps.edn file and see “only deps” in there … #2018-06-1803:05cflemingI guess I can always have a cursive.properties file alongside the deps.edn, but that seems way uglier to me.#2018-06-1803:05cflemingWell, except it’s not only deps.#2018-06-1803:05cflemingI mean, there are tasks with aliases etc etc as well.#2018-06-1803:05cflemingAnyway, I’ll wait and see what Alex says.#2018-06-1803:07hagmonkThat was my thought too, I think it’s more accurate to say it contains “keys required to build a classpath and invoke the JVM with that classpath”#2018-06-1803:11seancorfieldThere's an argument that Leiningen's project.clj sets a precedent for allowing additional tool-specific keys...#2018-06-1803:15hagmonkI think tools.deps is a deliberate move away from that - where is the sweet spot though? Should the canonical name for the project be something in the deps file? It’s not really a tool-specific thing at all. Ironically, the deps file is a list of canonical names for other projects, or at least the names they go by in their repos #2018-06-1803:21seancorfieldI would say that deps.edn goes a long way toward removing the importance of names -- you can give arbitrary names to local and Git deps. The names of Maven deps really only exist to match up with group/artifact names that are published. When you depend on, say, a Gist, or some code in a random directory, the name is unimportant and synthetic for the most part...#2018-06-1803:22cflemingSadly, my users are not going to want shas as their module names.#2018-06-1803:25cflemingThey’re fine in a gist link, but not for a descriptive name for someone who wants to get an overview of their project.#2018-06-1803:33hagmonkThese are the kind of highfalutin customers @cfleming's software attracts :)#2018-06-1803:36cflemingThey’re so fussy.#2018-06-1803:39hagmonkFor some reason I'm reminded of Ghostbusters: "I've worked in the private sector … they expect results"#2018-06-1803:41cflemingIt does sometimes help to imagine myself as Bill Murray when I have to do this stuff.#2018-06-1904:26cflemingIs there a list of likely locations I can try to look for the clojure command, so I can default it for the user if it exists?#2018-06-1904:26cflemingCurrently my list only contains /usr/local/bin/clojure from OSX.#2018-06-1904:27cflemingWhere is it on linux or WSL?#2018-06-1904:31richiardiandrea@cfleming not sure about windows but I guess on Linux it really depends, I actually added a prefix option to make install location customizable. Usually you use something like which or env depending on the use case. This answer might help you: https://unix.stackexchange.com/questions/12736/how-does-usr-bin-env-know-which-program-to-use#12749#2018-06-1904:31cfleming@richiardiandrea Thanks. The problem is that when I execute from within IntelliJ, the user’s path is generally not set correctly.#2018-06-1904:32cflemingSo the user will have to configure it, but I was hoping for a list of likely absolute paths I could check.#2018-06-1904:33richiardiandreaThe one you mentioned is most likely one. I use $HOME/.local/bin/clojure#2018-06-1904:33richiardiandreaSome distro might install in /opt#2018-06-1904:34richiardiandreaSome directly in /usr/bin maybe? I guess these are the most common if I were to answer that#2018-06-1904:42cflemingHow many installers for linux distros are out there at the moment?#2018-06-1905:17seancorfield@cfleming Why is the user's path not set for IntelliJ? (genuinely curious -- I'm not used to programs running without the standard user shell environment being available... I think?)#2018-06-1906:43dottedmag@seancorfield I have stumbled upon it on Mac: Dock and Finder are launched directly by user's launchd, which is launched directly by system launchd, so no shell is executed or profile file is read, so their environment is pretty bare.#2018-06-1906:44dottedmag@cfleming /opt/local/bin/clojure for MacPorts#2018-06-1906:46dottedmag@cfleming /usr/bin/clojure on Arch Linux (and I suppose in every other distro that bothered to package Clojure)#2018-06-1906:48dottedmag@cfleming /usr/local/bin/clojure for Linux installer from http://clojure.org#2018-06-1907:45cfleming@dottedmag Thanks!#2018-06-1907:46dominicmvoid places it in /usr/bin/clojure too#2018-06-1909:27cfleming@dominicm Is that on linux?#2018-06-1909:39dominicm@cfleming yes#2018-06-1911:38mfikes@cfleming FWIW, lein-tools-deps went with a mixture of overridable defaults along with searching https://github.com/RickMoynihan/lein-tools-deps/blob/master/src/lein_tools_deps/env.clj#L14-L22#2018-06-1911:40mfikesGiven the above, it would ultimately end up having defaults like this on non-Windows platforms
["/usr/local/bin/clojure" "/usr/bin/clojure" "/opt/local/bin/clojure"]
#2018-06-1912:12rickmoynihanYeah I’d much rather of not relied on the executable at all (and just used the library); but then you don’t know where the find the :install deps.edn file#2018-06-1912:43alexmillerI’m considering moving that into tools.deps#2018-06-1916:43rickmoynihanso it would be on the classpath?#2018-06-1916:45dominicmThe risk with that I guess is that organisations cannot specify repos globally. Maybe the root directory should go into /etc/clojure.conf or something. That's what apache, nginx, etc. do#2018-06-1917:02alexmillerit would be on the classpath or perhaps it would be automatically included such that it isn’t even a distinct deps.edn. still tbd.#2018-06-1917:03alexmiller@dominicm I think that would be the right solution to that problem, if we wanted to do something about it (which I do not at this time)#2018-06-1923:02cfleming@rickmoynihan Yes, I would much prefer that too - then the user could just specify a tools.deps version.#2018-06-1923:02cfleming@alexmiller Moving the install file into tools.deps would be great.#2018-06-1923:03cflemingOne possible source of confusion with that is that the versioning is different for tools.deps and the cli stuff.#2018-06-1923:20cfleming@mfikes Thanks - I looked through the lein-tools-deps issues when researching this but hadn’t found the code.#2018-06-2016:09gfredericksI just encountered my first instance of a clojure library only being available via github#2018-06-2016:11nwjsmith?#2018-06-2016:14gfredericksi.e., it's not on clojars#2018-06-2016:14gfredericksyou are expected to use it by specifying the github repo and a sha#2018-06-2016:53seancorfieldThe Cognitect Labs test-runner is like that -- but I assume you mean "aside from clj-specific tooling"?#2018-06-2016:54seancorfield(also my clj-new is not on Clojars but, again, clj-specific tooling)#2018-06-2018:32kennyCan I specify a username and password for a Maven repo?#2018-06-2018:57seancorfield@kenny In settings.xml I believe...#2018-06-2018:58kennyNot in a deps.edn?#2018-06-2018:58seancorfieldYou certainly don't want your Maven creds in your project's deps file under version control.#2018-06-2018:58kennyVia environment variables.#2018-06-2018:59seancorfielddeps.edn is static EDN, no variables.#2018-06-2018:59kennySo there is no way to use private Maven repositories? Or does tools-deps read in settings.xml?#2018-06-2019:01seancorfield@kenny Discussed yesterday in #clojure https://clojurians.slack.com/archives/C03S1KBA2/p1529458093000094#2018-06-2019:01seancorfieldI haven't tried it myself (and haven't seen specific documentation about it).#2018-06-2019:13seancorfield@kenny I haven't read through all of the details but this is the JIRA issue that covers this https://dev.clojure.org/jira/browse/TDEPS-9#2018-06-2019:14seancorfieldLooks like it relies on Maven's standard settings.xml file according to the patch in that ticket.#2018-06-2019:15kennyGot it, thanks!#2018-06-2019:54alexmilleryes, that is all correct#2018-06-2019:54seancorfield(it was a TIL moment for me so thank you for asking @kenny!)#2018-06-2019:54alexmillerFor Maven deps in authenticated repositories, existing Maven infrastructure is used to convey credentials via the ~/.m2/settings.xml:
<servers>
…
<server>
<id>my-auth-repo</id>
<username>zango</username>
<password>123</password>
</server>
…
</servers>
Then in your deps.edn include a repo with a name matching the <id>:
{:deps
{authenticated/dep {:mvn/version "1.2.3"}}
:mvn/repos
{"my-auth-repo" {:url ""}}}
#2018-06-2020:04alexmillerI added this info to docs: https://clojure.org/reference/deps_and_cli#_maven_authenticated_repos#2018-06-2020:28seancorfieldThanks. I'd searched for settings.xml earlier and couldn't find anything in the guides or reference.#2018-06-2020:58alexmillerI forgot to add it#2018-06-2100:44kennyThanks!#2018-06-2019:55hagmonkis it not possible to specify relative paths in :local/root?#2018-06-2019:56alexmillerit is possible#2018-06-2019:56hagmonkI'm trying to use lein-tools-deps to point to two different deps.edn, one of which points at the other via relative path#2018-06-2019:56hagmonkcould be a lein-tools-deps bug …#2018-06-2019:57alexmillerthere are some known issues with local to local and relative paths#2018-06-2019:57alexmillerparticularly around :paths#2018-06-2019:58cfleming@alexmiller Just a gentle reminder about the module naming issue, you said you had had discussions about this in the context of git deps.#2018-06-2020:01alexmilleryeah, haven’t had time, and won’t today. but I haven’t completely forgotten#2018-06-2020:01cflemingOk, thanks#2018-06-2020:02hagmonkchanging :local/root to be an absolute path resolved things for lein-tools-deps. Hum.#2018-06-2020:09hagmonkOh cute, I got it working#2018-06-2020:10hagmonkSolution is to not specify the deps.edn files directly in project.clj, just leave it as :project. Then, make an uber deps.edn that points at the deps.edn files. Things resolve as expected#2018-06-2122:28rickmoynihanGlad you got this working; if you think it’s a bug though please file an issue with lein-tools-deps
As Alex says, there’s some open bugs on tools.deps itself to do with paths, so could be that; but could also be due to us resolving paths relative to :project… would need to have a test case to recreate#2018-06-2115:21timgilbertSay, is there a tools.deps equivalent of lein deps, meaning "download everything you need to so I can cache it on my CI server"?#2018-06-2115:26bronsaclj -Sforce < /dev/null should do it#2018-06-2115:29sparkofreasonI'm running most of my dev CLI stuff under WSL, but Cursive runs under Windows. So a tool like lein-tools-deps won't work correctly in the Cursive context. Anybody have any luck working around this?#2018-06-2115:44richiardiandreaOh. I hoped that would work for a couple of colleagues of mine. I have exactly the same setup for macos users#2018-06-2116:02hagmonkI think lein-tools-deps works by invoking the clj command? If so you’d need to wait for Windows port. #2018-06-2116:06sparkofreasonI can get clj to execute under WSL using a batch script, but not sure that will be sufficient.#2018-06-2116:15seancorfield@U06HHF230 would have more insight but I would have expected lein-tools-deps to work on Windows if you explicitly specify which deps.edn files you need to use in project.clj so it didn't have to rely on tools.deps.alpha's ability to shell out to the clojure command?#2018-06-2116:43richiardiandreaI think lein-tools-deps calls clojure under the hood, but I saw that you can configure the location of it as well.#2018-06-2117:13seancorfieldWell, if it asks tools.deps.alpha for the (clojure-env), the shell out to clojure happens there (inside tools.deps.alpha itself). When I switch development machines (fairly soon, I suspect), I'll also be dealing with this since I use Atom/ProtoREPL on the host O/S. Right now, I'm on a Mac so the host for Atom is the same as the host for lein/`boot`/`clj` but I've already run into this on (Windows) laptop and my workaround was to start a REPL under WSL and then connect to it from Atom -- but there are still gotchas like file paths differing between Windows and WSL: when you "load file", ProtoREPL assumes that the REPL can be asked to load the file using the editor host path which fails unless you've been very careful setting up symlinks in WSL etc.#2018-06-2122:45rickmoynihanHi, sorry I don’t use Windows or have access to a machine I can test this setup on. Am I right that the problem is that you’re effectively running clojure in a (pseudo) linux environment, but Cursive in Windows; so the paths are resolving incorrectly?
@U04V70XH6 is right that we call clojure-env and rely on tools.deps shelling out to clojure to get the env for us. Unfortunately it’s a slightly tricky problem just now and the current design is a bit brittle.
I’d be open to disabling the call to clojure-env; though if we do that you’d also need to set the paths to your :install deps.edn and user one by hand.
If this issue is effecting you though, please open an issue where we can discuss more.#2018-06-2200:53cflemingI was actually planning to ask about this. It is possible to access WSL from Windows? If I try to open /usr/local/bin/clojure or whatever from Windows, does that work or not?#2018-06-2200:54cflemingMy currently plan for Cursive is to bypass the CLI stuff altogether, and just use the library. There are some outstanding questions about how to do that, such as how to get the system deps.edn - for an initial beta I’m going to bundle it with Cursive, but I’m hoping that it will eventually be available in the lib - Alex mentioned he was thinking about that.#2018-06-2200:55cflemingAssuming I just use the lib, then it should all work on Windows.#2018-06-2200:56cflemingAnother complication is that the user will have to choose the version of t.d.a to use, and those versions don’t correspond to CLI versions, so there’s some potential for confusion.#2018-06-2200:57cflemingAnd I guess if people are using the CLI in WSL and Cursive can’t access that, then that will also be confusing since you’ll end up with two parallel systems.#2018-06-2201:00cflemingThis is old, but it has big red letters: https://blogs.msdn.microsoft.com/commandline/2016/11/17/do-not-change-linux-files-using-windows-apps-and-tools/#2018-06-2201:01cflemingThe red letters are:
> DO NOT, under ANY circumstances, create and/or modify Linux files using Windows apps, tools, scripts, consoles, etc.
And some following large black ones:
> Creating/changing Linux files from Windows will likely result in data corruption and/or damage your Linux environment requiring you to uninstall & reinstall your distro!#2018-06-2201:02cflemingSo, I read that to mean that using the CLI from WSL will be totally inaccessible to Cursive anyway.#2018-06-2201:02cfleming@U066LQXPZ ^^#2018-06-2201:10sparkofreasonI think that guidance is somewhat out of date, and the interop between WSL and Windows file systems is much improved. I actually keep all the clojure project stuff on the Windows side anyway, but and do CLI stuff from WSL. #2018-06-2201:13cflemingRight, but I think that WSL is aware of Windows but not the other way around.#2018-06-2201:14cflemingAlthough I haven’t really followed it recently.#2018-06-2201:14sparkofreasonIt is possible to execute clojure under WSL from Windows, bash -c "clojure". I haven't been able to make that work with lein-tools-deps yet, was planning on making a fork and investigating locally. #2018-06-2201:15cflemingI’d be selfishly interested to know if you could execute it via ProcessBuilder from a Java app running on Windows.#2018-06-2201:16cflemingIf that is possible, then it should be possible to access it from Cursive. I’m still probably going to go lib-only unless there’s a really good reason not to because it’s much simpler, but it would be interesting to know.#2018-06-2201:18cfleming@U066LQXPZ BTW that guidance is still in the WSL FAQ: https://docs.microsoft.com/en-us/windows/wsl/faq
> One of the main limitations of using WSL is that changing Linux files with a Windows app or tool is not allowed.#2018-06-2201:22cflemingComment here: https://superuser.com/a/1111092
> Every file that resides on a Volfs folder (like /home) has Extended Attributes that store the Linux permissions of that file. If you edit that file on a windows editor then those attributes are lost and file will disappear from bash.#2018-06-2201:28cflemingLooks like CLion works around this by running development over ssh: https://blog.jetbrains.com/clion/2018/01/clion-and-linux-toolchain-on-windows-are-now-friends/#2018-06-2201:31cfleming@U066LQXPZ If you’re running your dev in WSL, doesn’t that mean that your source files are also under WSL? How does IntelliJ access them? Or do you treat WSL as a server you deploy to?#2018-06-2201:41sparkofreasonAll of the files are in the windows FS. The tooling (apart from the IDE) executes under WSL, and can work with the windows FS no problem. #2018-06-2201:41cflemingOh right, of course. So your Maven local repo etc is all under Windows? That makes sense.#2018-06-2201:42cflemingSo I guess the only issue would be accessing the installed deps.edn files (system and user). Do you use the CLJ_CONFIG env var to keep the user one under Windows too?#2018-06-2201:51sparkofreasonNo, that is on the todo list. #2018-06-2201:52sparkofreasonIt's how I have both lein and boot set up. #2018-06-2202:19sparkofreason@U0567Q30W (clojure.java.shell/sh "clojure.bat" "-Sdescribe") works. I assume ProcessBuilder would as well.#2018-06-2202:20cfleming@U066LQXPZ Ok, interesting, thanks.#2018-06-2202:21cflemingI guess the hard thing is that the paths that -Sdescribe will return are all in linux-land. So even if you configure CLJ_CONFIG to /mnt/c/Users/dave/.clojure or something, that path is still invalid in Windows.#2018-06-2202:25sparkofreasonRight, you'd definitely want to move it to be under windows. You can configure WSL to behave as though Windows drives are mounted under root, so you'd get /c/Users/dave/.clojure, but obviously that still can't be used as-is.#2018-06-2202:33cflemingI think it would be better to leave them under /mnt, since Cursive has to be aware it’s talking to WSL anyway.#2018-06-2202:33cflemingIt would make it easier to detect that it’s getting a Windows path back from Linux#2018-06-2203:17sparkofreasonYeah, but other things like docker benefit from this (so they tell me, I haven't run into this myself). Maybe some sort of config option to indicate WSL? #2018-06-2203:37seancorfieldMicrosoft Windows [Version 10.0.17692.1000]
(c) 2018 Microsoft Corporation. All rights reserved.
C:\Users\seanc>wsl.exe clojure
Clojure 1.9.0
user=>
C:\Users\seanc>wsl.exe clojure -Sdescribe
{:version "1.9.0.381"
:config-files ["/usr/local/lib/clojure/deps.edn" "/home/seanc/.clojure/deps.edn" ]
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/seanc/.clojure"
:cache-dir "/home/seanc/.clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
C:\Users\seanc>#2018-06-2203:37seancorfieldUse wsl.exe to invoke Linux commands 🙂#2018-06-2203:39seancorfieldAnd just to be clear, when you do that, the Linux command executes in the context of Linux, e.g., user.home: C:\Users\seanc>wsl.exe clojure
Clojure 1.9.0
user=> (System/getProperty "user.home")
"/home/seanc"
user=>
#2018-06-2204:01seancorfield(of course this totally doesn't help in terms of actually accessing those files from Windows, nor in starting/running a REPL that can accept "load file" commands from Windows)#2018-06-2204:02cfleming@U04V70XH6 Thanks. So I’d need to make Cursive WSL-aware.#2018-06-2204:02cflemingThe problem is also still accessing the deps files from Windows.#2018-06-2204:02cfleming(or, a problem, rather)#2018-06-2204:03seancorfieldYeah, it's a tough nut to crack. I've dinked around a bit with Windows-Atom/ProtoREPL and WSL-REPL but not really gotten anywhere.#2018-06-2204:03cflemingI think just using the lib and keeping everything in Windows-land looks like the cleanest solution.#2018-06-2204:04seancorfieldWell, once the lib knows how to -Sdescribe the environment...#2018-06-2204:05cflemingRight, I’d have to hack that for the moment, probably bundle the system deps.edn in Cursive, and perhaps allow the user to edit it somehow.#2018-06-2204:06cflemingAnd allow them to configure where their user one is, defaulting as required.#2018-06-2204:06seancorfieldMy fallback with Boot has been to have Boot installed on both Windows and WSL and let Atom start a REPL via Boot and keep all the files in a project structure under the C: drive that I can easily map to Linux (symlink to make the same file paths without /mnt/c).#2018-06-2204:09seancorfieldWhen the PowerShell port of clojure hits, we'll all be a lot better off -- but we'll still have the weird "duplication" of the "home" deps.edn because that will be in a different place on Windows and WSL. I can't see a workaround for that since the user home folder has to be different.#2018-06-2204:16cflemingYou could use CLJ_CONFIG in WSL?#2018-06-2204:17cflemingAnd point the linux one to where it is on Windows?#2018-06-2204:23seancorfieldI haven't tried CLJ_CONFIG...#2018-06-2204:26seancorfieldOh right, so on WSL, CLJ_CONFIG could point to /mnt/c/Users/<whatever>/... yeah, sounds like a good approach once we have PowerShell clojure.#2018-06-2204:38cflemingRight.#2018-06-2204:39cflemingAlthough generally I think the best option for clients like Cursive and lein-tools-deps would be bundling the system deps.edn in the lib, and perhaps providing a method to locate the user deps.edn.#2018-06-2204:54seancorfieldWhen I first built boot-tools-deps I baked in the deps.edn from the brew-install GitHub repo so I didn't need clojure installed on the system. Alex regularly told me that was a bad idea 🙂 So I changed it to call (clojure-env) once t.d.a had that API. And now we're talking about the system deps.edn being baked into t.d.a anyway! :rolling_on_the_floor_laughing:#2018-06-2210:53rickmoynihanInteresting discussion - I think I’d like to remove the call to clojure-env from lein-tools-deps and follow the bundled system approach; and just let people override the :system location if they need to.#2018-06-2219:52cfleming@U06HHF230 Yes, I agree - this would make Cursive much easier too#2018-06-2219:53cflemingThe only issue is the need to specify the version to use.#2018-06-2115:33alexmiller@timgilbert additionally, you could set :mvn/local-repo to a fresh directory, then capture that directory to know what you need.#2018-06-2117:12timgilbertAm I crazy, or is tools.deps super slow at downloading versus lein deps? I'm seeing my download step go from about 40 secs to about 4 minutes on the same hardware#2018-06-2117:43alexmillerI haven’t heard that before now#2018-06-2117:44alexmillerAre you comparing with cold m2 repo in both cases?#2018-06-2117:47timgilbertYeah, they are both running from nothing on circleci servers. I do wonder if it might have to do with invalid artifacts in my tree, combined with a private S3 repo.#2018-06-2117:48timgilbertI will do some poking around and file a ticket if I can come up with some numbers and a reproducible case#2018-06-2117:50alexmillerYeah, would be curious#2018-06-2117:50alexmillerSince most people have stuff in cache, most people wouldn’t notice#2018-06-2117:52timgilbertYeah. Does tools.deps cache misses? Somehow or another I've wound up with com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml in my deps tree, which doesn't exist, and it seems to be causing some of the latency along with several other similar artifacts#2018-06-2117:53timgilbert(Obviously, these are things I am going to exclude from the tree once I've tracked them down)#2018-06-2118:09alexmillertools.deps uses the same Maven apis that lein (via pomegranate) does to download metadata and deps#2018-06-2118:11alexmillerso I would expect it to generally do pretty similar things. there are some differences, notably that tools.deps uses its own transitive dep walk and can potentially choose different versions and may walk versions that lein deps doesn’t.#2018-06-2122:48rickmoynihanI’ve been meaning to ask are the differences in resolution documented anywhere, or formally known? Asking because it’s relevant to leiningen projects moving deps into deps.edn and lein-tools-deps.#2018-06-2200:54alexmillerWell it’s known to me but I haven’t gotten around to writing it up yet#2018-06-2200:55alexmillerIt’s somewhat similar to the new golang stuff #2018-06-2118:11alexmillertools.deps itself doesn’t explicitly do any caching of metadata or libs at all - Maven handles all of that#2018-06-2118:13alexmillerseveral possible things to investigate there. if you file a ticket with a deps.edn to repro, I would eventually take a look at it#2018-06-2118:13timgilbertOk, cool. I'll see what I can do to come up with a reproducible case. Thanks!#2018-06-2121:45kennyIf both :sha and :tag are provided for a Git dep but :tag points to a different :sha, which SHA is used?#2018-06-2121:56seancorfield@kenny I thought only :sha was used for dependencies and the :tag was there for use with the option that adds :sha based on :tag...?#2018-06-2121:56seancorfield(it's easy enough to test the behavior!)#2018-06-2121:57kennyThis part from https://clojure.org/reference/deps_and_cli#_dependencies makes it a bit unclear
> The :sha attribute should indicate the full commit sha
> The :tag attribute is optional but can be used to indicate the semantics of the sha#2018-06-2121:58kennyAlso makes it sound like :sha is always required.#2018-06-2121:59kennyYes. Will test it out later today. Definitely could use some clarification in the docs though.#2018-06-2122:04seancorfieldThat reads that :sha is used for dependencies and :tag is purely descriptive -- although there's an option to update deps.edn adding/replacing SHAs with the values that correspond to tags: clj -Sresolve-tags#2018-06-2122:06seancorfieldMy interpretation of that is that if you have :sha and :tag and the latter doesn't match the former, it would not affect resolution. And if you ran clj -Sresolve-tags it would update the :sha to match what the :tag pointed at. @alexmiller can you confirm or deny?#2018-06-2122:06alexmillerCorrect#2018-06-2122:07kennyThat interpretation makes sense to me. I'd still suggest adding that information into the docs 🙂#2018-06-2122:11alexmillerStill considering whether to remove :tag and -Sresolve-tags entirely :)#2018-06-2122:19kennyThe reason I was thinking about using :tag is to convey breakage. With only :sha, determining breakage is a lot harder.#2018-06-2201:11alexmillerCan you explain?#2018-06-2201:11alexmillerIf you mean like api breakage, just don’t do that :)#2018-06-2201:12alexmillerWe have sha logic comparison in there - it can tell which one is a parent of the other#2018-06-2201:12alexmillerIf more than one available in the tree, it will pick the descendant #2018-06-2215:30val_waeselynckIt seems to me it's not possible to have a spaces in a :main-opts entry (e.g :main-opts ["--space-separated-arg" "Joe Jack Williams Averell"]), is this is known issue?#2018-06-2215:32alexmillerYes, use commas for now#2018-06-2215:33alexmillerOr in a case like this, not going to work for now#2018-06-2215:38val_waeselynck@alexmiller thanks, currently my workaround is simply to pass the args form the command line#2018-06-2217:16seancorfield@val_waeselynck https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L32-L37 commas-as-spaces#2018-06-2404:40hagmonkI've updated https://github.com/hagmonk/find-deps, it now allows you to save matches even if the deps.edn file doesn't exist (handy for new projects). It also has a new -F:cli output format that prints the deps on a single line appropriate for clojure -Sdeps usage#2018-06-2408:58pesterhazy@hagmonk just tried it, it doesn't seem to work for me. After following the instructions, clj -A:find-deps -h gives me the normal "deps" help screen#2018-06-2409:00pesterhazyis the sha in the readme up to date?#2018-06-2413:03alexmillerDo you have the latest clj? That -h thing was a bug I fixed a while back#2018-06-2414:29pesterhazy@alexmiller I have version = 1.9.0.381 from homebrew#2018-06-2414:31alexmillerThat should be latest#2018-06-2414:33alexmillerMaybe you’d see that if it’s missing the find-deps alias#2018-06-2415:26pesterhazy@alexmiller that was it. I followed the readme but not correctly. PR to make the docs more explicit for busy readers: https://github.com/hagmonk/find-deps/pull/1#2018-06-2517:44hagmonkI've pushed a new find-deps with @pesterhazy's PR merged and a fix for no longer returning SNAPSHOT versions from clojars (hard coded for the time being)#2018-06-2517:48pesterhazy@hagmonk just tried it, works really well#2018-06-2517:48pesterhazyI think hardcoding the hide-SNAPSHOT feature is a much better default#2018-06-2517:49hagmonkYes definitely, I think the bug should not crop up in maven searches, I need to spend time finding an example there to validate that#2018-06-2517:49hagmonkI believe maven doesn't offer snapshot results by default#2018-06-2517:53pesterhazygood job on the library btw!#2018-06-2517:59hagmonkThanks! Let me know if any of the search results don't look good :)#2018-06-2518:54dominicm@hagmonk one piece of feedback is that it will modify the deps.edn formatting. #2018-06-2518:56hagmonk@dominicm that's worth an issue on github … appreciate any pointers you might have for how to avoid that#2018-06-2518:57dominicmPack avoids it (my library). Happy to try share a library for solving this problem. I'm secretly hopeful it can go into tools deps itself. #2018-06-2519:27hagmonkThanks @dominicm, I'll have a look after work#2018-06-2519:28hagmonkI'm still enjoying how much time this is saving me, occasionally I'm surprised it works as well as it does 😂#2018-06-2519:28hagmonkclj -A:find-deps "clojure cache"
{:deps {org.clojure/core.cache {:mvn/version "0.7.1"}}}
#2018-06-2519:31pesterhazyone possibility would be to run the result through zprint, a great library for re-formatting clojure code or edn#2018-06-2519:32pesterhazyzprint is an under-appreciated gem#2018-06-2519:55dominicm@pesterhazy the problem is that everybody does formatting a little differently. I took the approach of trying to match existing indentation at most levels. #2018-06-2520:12mkvlrit would be so great if we could agree on a canonical way to format clojure code#2018-06-2520:13mkvlrhaving worked with elm-format and exfmt I found it so nice to just not have to worry about this at all anymore#2018-06-2520:14pesterhazyzprint gets you 90% there#2018-06-2520:16dominicmZprint puts my defn all on one line, which feels wrong#2018-06-2520:23pesterhazyThey are too short :)#2018-06-2520:24dominicmDo you happen to know of a way to change that behaviour? 80 chars isn't a goal, so much as a limit.#2018-06-2520:27pesterhazyNot off the top of my head#2018-06-2520:29pesterhazyBut if you get consistency, convenience and the end of style wars in return, wouldn’t oneliners be a small price to pay?#2018-06-2520:30dominicmI want consistency, very much. But I first have to convince important people that they will like the formatting, and that it's worth pioneering.#2018-06-2607:44kommenthe :respect-nl config should be able to help you with “80 chars isn’t a goal”: https://github.com/kkinnear/zprint/#respect-nl-false#2018-06-2607:50kommenit says it is meant for vectors, but might work for lists as well#2018-06-2607:51dominicmI don't necessarily want to have it retain where I place newlines. I just find it weird to see:
(defn foo [args] body)
over
(defn foo
[args]
body)
#2018-06-2607:59kommenah, I see. haven’t tried to configure that, but pretty sure it is possible#2018-06-2608:04dominicmIt would be nice to have a style which is "do what most libraries do for formatting"#2018-06-2608:07kommenI think the :community style is meant for that. kkinnear has been open to adapt that style to fit best practices, especially if those things would be documented in the clojure community style guide#2018-06-2608:50pesterhazyNOT respecting newlines is one of the best things about elm-format#2018-06-2608:51pesterhazyyou can totally write your code in a single line and elm-format will expand it out#2018-06-2608:51pesterhazyit helped me a lot when I was learning the language#2018-06-2608:55dominicmI particularly don't want my tool to look at the input beyond raw clojure forms tbh. I'd even be happy if it made decisions about #:foo {:blah 10} into {:foo/blah 10} (but that's because I don't like the syntax)#2018-06-2520:14pesterhazyAgree about elm-format, it’s a pleasure to use #2018-06-2520:14mkvlrwe’re using zprint but there’s so many knobs. I think an autoformatter should ideally have 0 options. (or maybe one for the line width)#2018-06-2520:15mkvlrsorry for going a bit offtopic here…#2018-06-2521:57lwhortoni’m not sure if this is the right place to ask… but does the :git/url coordinate require a deps.edn file at that coordinate?#2018-06-2521:57lwhortoni.e. it doesn’t try to read a project.clj or boot file… it only plays nicely with another deps.edn (or maybe a pom) file?#2018-06-2521:58ghadiThat's right @lwhorton #2018-06-2521:58ghadiIt does read deps.edn and pom.xml transitively#2018-06-2522:08alexmillercorrect (for now)#2018-06-2522:08alexmillerit’s also tolerant of finding none of those for the case of a dep with no transitive deps#2018-06-2522:56cfleming> I think an autoformatter should ideally have 0 options. (or maybe one for the line width)
Preach it! If I ever design a language, it will have a gofmt-style formatter from day 1.#2018-06-2522:57cflemingIndentation turns out to be one of those things that everyone thinks they care about, until they can’t control it, at which point it turns out they don’t care much as long as the results are reasonable.#2018-06-2522:59alexmiller+1#2018-06-2602:12hagmonkWould love for us to swallow that frog and just settle on one format to rule them all. I'd gladly give up my personal formatting preferences in exchange for not having to care about this ever again :)#2018-06-2606:44pesterhazyIf @bozhidar and @cfleming band together and settle on a zformat formatting style, that’s a significant portion of the market#2018-06-2606:45pesterhazyMake it opt-in and apply on save#2018-06-2606:51orestisWe’re considering introducing Clojure to an existing team – not having to argue about formatting would be huge — zprint looks fantastic, with native-image support so could even be run on a github pre-commit hook.#2018-06-2606:54hagmonkI think people quite quickly get the whole paredit/parinfer concept, where the editor prevents you from creating an improperly delimited program … it doesn't seem a fair stretch to also prevent you from ever creating an improperly indented program#2018-07-0208:46rickmoynihanI’m definitely all for structured editing; a classic example of a really good one was Diva Scheme for PLT Scheme (now Racket):
https://www.youtube.com/watch?v=GnQV4je9wTQ
You need a mode to disable them temporarily though e.g. when resolving a merge.#2018-06-2606:56hagmonkI'm trying out parinfer and am irritated that I seem able to hit "tab" and move blocks of code around to create valid but aesthetically awful code (probably some config thing I borked). I'd love for it to be impossible to write untidy code, like making it physically impossible to create:#2018-06-2606:57hagmonkdangling paren:
(comment
(some-stuff "test")
)
or have two newlines between def:
(def this "thing")
(def other "thing")
… etc#2018-06-2607:00dominicmParinfer + zprint could be quite great for beginners... #2018-06-2607:16orestisOTOH, I tried Parinfer via Nightcode — I knew how to write Clojure code in spacemacs via Cider, which doesn’t try to do too much — but parinfer messed me up quite a lot — it tried to be helpful but got in my way.#2018-06-2607:35dominicmhave you tried it since smart mode was added?#2018-06-2607:35dominicmhttps://shaunlebron.github.io/parinfer/demo.html#2018-06-2609:16orestisI need to find how to integrate with spacemacs and evil-mode, but will try again…#2018-06-2607:17orestisIt helps a beginner to be able to balance forms manually, esp. if you have a linter that will point out possible problems, and a formatter that would take stylistic decisions away from you. Parinfer might be useful once you make relatively few mistakes when coding Clojure.#2018-06-2607:24hagmonkParinfer is a little more loose, I agree. I think we could go a step further and have editors that totally precluded interim mistakes at all. Like: trying to add a third argument to a 2-arity function. Why do we let you type that in? Why do we let you type in something that is highly likely knowable at the reader level to be invalid, such as transposing the arguments to map – i.e. putting something that doesn't support IFn as the first argument?#2018-06-2612:09alexmillerThere’s an interesting talk in this area at Strange Loop this year. https://thestrangeloop.com/2018/hazel-a-live-functional-programming-environment-with-typed-holes.html#2018-06-2607:26hagmonkI realize the dynamic nature makes some things hard to guarantee, but even though I'd say I'm a journeyman Clojure programmer, I would still appreciate the help. It's like type checking but while you type, not when you compile.#2018-06-2607:27orestisThe UX for that is extremely hard to get right though — when I type a character, I expect it to show up in my buffer. If the editor puts a squiggly line later, then I know something’s wrong and I can learn something from it.#2018-06-2607:31hagmonkIt need not be "can't type the key" levels of strictness, more like how we flag unbalanced parens today … "this is for sure going to barf". Ideally unbalanced parens become physically impossible, while messing up arguments graduates to being detected in the editor buffer rather than runtime or at compilation#2018-06-2607:35hagmonkI think we also have opportunity (I realize this is drifting off tools.deps but ya'll are an erudite bunch) for hinting, for instance, when I type (let [[foo bar] some-thing] …) I occasionally pass in something non-seqable as some-thing and invariably it takes me a few minutes to reverse engineer the cryptic error message and wth I did to cause it. Just flagging in the editor "reminder: some-thing now has to be seqable!" could be quite interesting …#2018-06-2607:41cfleming@hagmonk This is what you’re looking for: https://www.facebook.com/notes/kent-beck/prune-a-code-editor-that-is-not-a-text-editor/1012061842160013#2018-06-2607:42cflemingPlus some static analysis#2018-06-2607:42cflemingThis should probably get moved over to #editors though.#2018-06-2607:45hagmonkYes, I recall this whizzing past at some point. I'll take another look#2018-06-2610:45mgrbytehaving trouble figuring out what the correct coordinate would be to use a local jar (datomic-pro), anyone doing this?#2018-06-2610:47dominicm@mgrbyte :local/root pointed at the jar#2018-06-2610:53mgrbytethat's what I've tried. doesn't work for me. I have:
:deps {
com.datomic/datomic.api {:local/root "/home/matt/datomic-pro/datomic-pro-0.9.5697/datomic-pro-0.9.5697.jar"}
}
#2018-06-2610:54dominicm@mgrbyte I may have crossed wires, just to check, can you try :deps/root instead?#2018-06-2610:54mgrbyteIt's definitely :local/root in the map in the docs#2018-06-2610:55dominicmhttps://clojure.org/guides/deps_and_cli#local_jar nope, I remembered correctly#2018-06-2610:55dominicmah, you're having an issue with that working 😄#2018-06-2610:55dominicm@mgrbyte I'm not sure I understand, what is the problem? 🙂#2018-06-2610:55mgrbyteclj works, but requiring the dep doesn't#2018-06-2610:55mgrbyteclj
Clojure 1.9.0
user=> (require '[datomic.api])
ClassNotFoundException org.fressian.handlers.IWriteHandlerLookup java.net.URLClassLoader.findClass (URLClassLoader.java:381)
user=>
#2018-06-2610:56mgrbyteI'm using the very same jar in a lein project, which works fine.#2018-06-2610:57mgrbyteinstalled locally via bin/maven-install in the datomic-pro distro (same path as the jar in the deps I listed above)#2018-06-2610:58mgrbytenm, think I see what's going on here#2018-06-2610:59mgrbytedatomic pro has a pom.xml with a bunch of deps in it. can't just use that jar, need all the deps in "./lib" from the distro too.#2018-06-2612:11alexmillerThe latest tools.deps should actually read the pom inside the jar and load those for you#2018-06-2612:12alexmillerMake sure you are on latest clj tool#2018-06-2612:54mgrbytethanks, will try that now#2018-06-2612:54mgrbyteIs there any reason to prefer to doing that than using bin/maven-install and using the mvn/version w/datomic-pro?#2018-06-2612:55alexmilleryou can do either. you can also use the clj support to load the pro jar from your authenticated s3 datomic repo#2018-06-2612:57mgrbytety#2018-06-2611:05dominicmI thought it might be that.#2018-06-2611:06dominicmYou should be able to set :local/root to the dir with the pom.xml in, but not certain.#2018-06-2611:45mgrbytenearly, fails with some dep. Not a problem tho, can just use :mvn/version of the local install#2018-06-2611:54hkuptyQuick question.. does tools.deps expose a function to add to classpath? I couldn't find anything from the documents but I would also like to avoid adding pomegranate or lucidity.package..#2018-06-2611:56borkdude@ingvij I think it’s called add-lib#2018-06-2611:56borkdude@ingvij http://insideclojure.org/2018/05/04/add-lib/#2018-06-2612:01hkuptyNot there as of latest published version..#2018-06-2612:01alexmillerStill experimental#2018-06-2612:01hkuptyI see.. So that can change until I actually can rely upon..#2018-06-2612:01hkuptyI'll wait then.. not in a hurry for that though#2018-06-2619:11hagmonk@alexmiller I was going to start thinking about add-lib in the context of find-deps. I could have users do this: (-> "clojure cache" find-deps.core/deps clojure.tools.deps.alpha.repl/add-lib)#2018-06-2619:12hagmonkor I could provide a fn inside find-deps.core that invokes add-lib for you. right now I'd have to depend on your add-lib branch, so I was hesitant to bake that in quite yet#2018-06-2619:12alexmilleryeah, I wouldn’t do that yet#2018-06-2619:14hagmonkcool, I might limit integration to some notes in the readme for the time being. I can't wait until we have a workflow where I can seamlessly add deps without restarting anything :)#2018-06-2620:41cfleming@hagmonk Have you ever tried alembic?#2018-06-2620:56hagmonk@cfleming I have not ... Googling ...#2018-06-2620:57hagmonkfascinating, this would have been nice to have :)#2018-06-2621:42alexmillernote that the tools.deps add-lib is a little different than the other existing things out there#2018-06-2621:43alexmillerit’s not just blindly adding a lib (and/or it’s transitive deps), it’s actually detecting which things you don’t already have, resolving lib versions, and updating an internal map of loaded libs and versions#2018-06-2621:44hagmonkand obviously it can work on more than just jars#2018-06-2621:45alexmillerthat too, all the standard tools.deps differences#2018-06-2622:09cflemingAlembic also does the detection of existing things you have installed, although it doesn’t use a map, it uses the list of things in your existing classloader.#2018-06-2622:48alexmillerThe tools.deps impl falls back to that#2018-06-2622:31hagmonkI have some crazy ideas in this space - I feel like we can go a lot further than just adding the libs. For find-deps I want to analyze clj code on github to build "recommended deps" based on the one you're importing. For instance, reagent and re-frame often go together. Intelligently figuring out exclusions could be another one.#2018-06-2622:46sparkofreasonHow would I do a local build and deploy of tools.deps.alpha?#2018-06-2622:46alexmillermvn clean install#2018-06-2622:47alexmillerThat will use the current snapshot version in the pom#2018-06-2623:26sparkofreasonDebugging some issues using tools.deps.alpha on Windows, and pretty much a noob with aether. Why would clojure.tools.deps.alpha.util.maven/make-system return nil?#2018-06-2623:26alexmillerDunno, not sure why windows would be special#2018-06-2623:27alexmillerI’ve got ports of clj using tdeps on Windows that work#2018-06-2623:29sparkofreasonHmmm, that's what I would have guessed. It's being called by lein-tools-deps, so maybe the problem originates farther up the call stack.#2018-06-2623:41seancorfield@dave.dixon Perhaps because lein-tools-deps calls (clojure-env) in t.d.a. and that shells out to call clojure to get the list of deps.edn files to consume -- and I would expect that command to fail on Windows (until we have a Windows port of the clj/`clojure` shell scripts).#2018-06-2623:41seancorfield(so maybe it doesn't have the environment that make-system requires)#2018-06-2623:42sparkofreason@seancorfield That I have working, albeit locally hacked. I'll double-check that the right info is getting scraped.#2018-06-2623:42seancorfieldAh, OK. Cool.#2018-06-2623:43seancorfieldI nearly said "...unless you have a mocked up version of clojure on Windows!" 🙂#2018-06-2701:13sparkofreasonOkay, wacky problem 1 solved, somehow I managed to have two versions of lein installed.#2018-06-2716:22sparkofreasonThanks to everyone's help, I was able to get a working setup using lein-tools-deps and Cursive on Windows, with clj installed on WSL. It's pretty hacky, but I think will tide me over until the tooling solves this problem in the right way. See fork on lein-tools-deps at https://github.com/ComputeSoftware/lein-tools-deps. Use at your own risk 😋#2018-06-2810:04darnokI'd like to use :local/root to tie up my dependent projects in the monolith repo, but I have an issue with one of the dependencies. It has one Java source file. Can you recommend, how to handle this?#2018-06-2811:33alexmillerWell one solution is to move it into its own project and make an artifact#2018-06-2811:34darnokok#2018-06-2811:37darnokI thought that it would need to be resolved like this. Thanks 🙂.#2018-06-2814:00dottedmag#2018-06-2814:01dottedmag@alexmiller ^ is this an intentional behaviour?#2018-06-2814:20alexmillerthere’s a ticket, send me a patch :)#2018-06-2814:21alexmillerI just haven’t gotten to it yet#2018-06-2814:22alexmilleractually, maybe nobody made a ticket so feel free to make one in https://dev.clojure.org/jira/browse/TDEPS#2018-06-2814:42pesterhazyyeah, the "problem" I reported a few days back was because the alias wasn't properly configured; a simple name check would caught this#2018-06-2814:49alexmillerhappy to fix#2018-06-2814:50dottedmag@bhauman ^#2018-06-2814:53bhaumanI think my use of the word extremely was extremely extreme#2018-06-2815:41seancorfieldIt's bitten me a couple of times so I'll vote that ticket 🙂 but it hasn't bitten me hard/often enough for me to summon up the energy to create such a ticket...#2018-06-2816:20hagmonktoday in #tools-deps: extremely extreme bugs! do they bite hard and often enough for a ticket? do they come from the Australian mainland? only jira can tell!#2018-06-2816:20hagmonk(clearly this channel is where all the action is at)#2018-06-2816:36alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-85 created#2018-06-2816:36alexmillervote away#2018-06-2816:36alexmillerif no else patches I’ll get to it when I next work on TDEPS#2018-06-2816:40bhaumanThanks Alex!#2018-06-2816:46seancorfieldYes, thank you, Alex!#2018-06-2821:07kennyWhy is the clojure CLI written in bash instead of Python or Node?#2018-06-2821:07dominicm@kenny because shell is universally available and python/node isn't.#2018-06-2821:08dominicmHopefully it can be ported to POSIX shell without any bash, that would be a nice improvement 🙂#2018-06-2821:09kennyWhy do you say Python and Node aren't universally available?#2018-06-2821:11dottedmag{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "12607d7d66527370777e"}, :content ("[email protected]")}#2018-06-2821:11dominicm@kenny because they're less likely to be installed than bash.#2018-06-2821:12dottedmagEven if they were, what'd be the point? clojure(1) is trivial.#2018-06-2821:15kennyInstalling Python/Node is trivial. Point is you get a script that is portable to any OS.#2018-06-2821:17dottedmag@dominicm clojure uses arrays from Bash. Not trivially convertable.#2018-06-2821:18dottedmagAlso that means testing the script under several implementations (busybox, dash and bash at least). Not worth it.#2018-06-2821:20dominicm@dottedmag that's unfortunate. Although I did stumble onto https://github.com/Ventto/libshlist recently.#2018-06-2821:22dominicmI personally am ready to remove bash for all but clojure on my system. I use dash & fish primarily.#2018-06-2821:22alexmillerbecause I wrote it and I don’t like Python or JS :)#2018-06-2821:22dominicm@alexmiller So 💎 ruby?#2018-06-2821:22alexmillergeez man#2018-06-2821:23alexmillerfor whatever reason, I just don’t get Ruby.#2018-06-2821:24dominicmOh really? Oh I was not expecting that response. I suppose you're not from the ruby shop that's associated with Clojure 😛. You strike me as a perl guy? 😉#2018-06-2821:24dominicm(I'll just keep guessing acting like I knew the whole time...)#2018-06-2821:24dottedmagI feel this is the exactly the kind of conversations that cause Codes of Conduct written which precludes snarky remarks and personal attacks.#2018-06-2821:24alexmillerI have used all of these languages profesionally#2018-06-2821:25dottedmag@dominicm The point of clojure is to just work. Anywhere. Without installing anything.#2018-06-2821:25alexmillerand I respect people that like and use all of them#2018-06-2821:25dottedmagThe script itself is trivial. There is no gain from using any other language.#2018-06-2821:25dominicm@dottedmag just to be clear, I actually hold no contempt for any of these languages. I feel like I have a fairly good grasp of where each one is pretty good.
I agree. I don't actually think ruby is suited here.#2018-06-2821:26alexmillerbash is about as lingua franca as it gets these days#2018-06-2821:26hagmonkbash is far and away the lowest common denominator on Unix-ish systems, with the possible exception of sh, which on many distros just links to bash.#2018-06-2821:26hagmonkit absolutely sucks, but if your goal is maximum multi-year portability, there's no other viable alternative#2018-06-2821:27dominicmbash isn't so bad.#2018-06-2821:27alexmillerI’ve grown to almost like it#2018-06-2821:27dominicmI wrote a whole panel in bash.#2018-06-2821:27dottedmagThis discussion already amounts to 30% of the size of clojure(1).#2018-06-2821:27alexmillerhaving done the windows port to powershell, that wasn’t bad either#2018-06-2821:28dottedmag@alexmiller Oh, there is PowerShell one? In the same repo?#2018-06-2821:28alexmillerit’s on a branch I think#2018-06-2821:28alexmillerif I pushed it#2018-06-2821:28hagmonkthere's a team at work that has >100k written in bash, with strict coding guidelines and practices it can work. You can grow to like many things if you immerse yourself in their idiosyncrasies :)#2018-06-2821:28alexmillerit’s not done yet#2018-06-2821:29dottedmagFound it, thanks#2018-06-2821:29dottedmagYeat, it beats cmd#2018-06-2821:30alexmillerfer sure#2018-06-2821:30alexmillerit’s kind of like if you took bash and applied sane engineering to it#2018-06-2821:30dottedmagAnd isane amount of syntax, but oh well.#2018-06-2821:31hagmonkheck, powershell does autocomplete of function arguments on the command line. It's kinda impressive#2018-06-2821:31kennyWhy not use ClojureScript compiled to Node?#2018-06-2821:31dominicm@kenny because node isn't always installed!#2018-06-2821:31kennyIt is trivial to install.#2018-06-2821:32dottedmagAmazing.#2018-06-2821:32hagmonk@kenny the goal is not to install anything except what is required to bootstrap clojure#2018-06-2821:32alexmillerstop trying to oppress me man#2018-06-2821:33hagmonkthis coming from a man comfortable writing all day in bash :)#2018-06-2821:33alexmillerthe water’s fine, come on in#2018-06-2821:33dominicm@hagmonk (and any others) I just pushed up 857bbc85c33b23b5f8e51d68eb724776564d9c16 to juxt/pack.alpha which adds skinny jar support, it spits it out in multiple forms based on the parameters. Any feedback welcome!#2018-06-2821:50dominicm@hagmonk let me know if this works for you btw.#2018-06-2821:50dominicmI don't really have a concrete use case myself yet, so I'm guessing at things a little. #2018-06-2821:57hagmonkyeah for sure, I'll poke at it when I get a chance!#2018-06-2915:18dominicmI had to do a little bump, having : in filenames doesn't go well with classpaths, 800afc066907e31fe90ce4df4f519e6224116867 is the latest commit#2018-07-0315:21dominicm@hagmonk Did you get any time to see if this looks good to you? 😄#2018-07-0323:50hagmonkSorry @dominicm, haven't forgotten, but I've been crushed at work lately :/#2018-07-0405:29dominicmNot at all. Entirely when you get some spare time. 😊 #2018-06-2821:34dominicmIt's in alpha-alpha, as it's not in master yet, I'll be rebasing at some point, but I won't force push over those commits.#2018-06-2821:34hagmonkI can't, I didn't quote that string properly and so I dived into an empty pool#2018-06-2821:35dottedmag@hagmonk rm -rf "${PREFIX}/${SHAREDIR}". Guess what happened? True story.#2018-06-2821:35kennyMake Node required to bootstrap Clojure.#2018-06-2821:41gfrederickson the off-chance that you're not trolling, that would be way too much friction#2018-06-2821:35dottedmag@hagmonk Postinstall script in a Linux package. Root privileges.#2018-06-2821:36hagmonkTruly a trap for young players, right there … a mistake you never forget :)#2018-06-2821:44hagmonkSo quick story time about the clojure installer being written in bash: the other day I was mucking with Homebridge on a raspberry pi, and needed to do some interrogation of a web API. In five minutes I had a Clojure REPL up with rebel readline, it was great. Had the installer required ruby, python, perl, etc, I would have had to dive into the Pi's package management and pull a ton of unrelated crap down, just to get a REPL. Had clojure needed a version of nodejs that was in any way incompatible with homebridge, I would have immediately faced the problem of juggling multiple npm versions. An example of the leverage you get from picking the unsexy, boring, lowest common denominator for bootstrapping tools#2018-06-2821:44dottedmagDoes PowerShell come preinstalled on Windows nowadays?#2018-06-2821:44hagmonkI do believe so#2018-06-2821:46alexmilleryeah#2018-06-2910:48cflemingOf particular interest is the git lib attached to library (“Deps: org.clojure/tools.gitlibs:cab6ab”)#2018-06-2910:49dominicmNice 🙂#2018-06-2910:50cflemingThere’s still lots of trickiness to work out, but the first 90% is done. Now there’s just the remaining 90%.#2018-06-2911:06Andreas LiljeqvistLooking forward to it, my temporary solution isn't exactly scalable...#2018-06-2913:49pesterhazy@cfleming this will be a godsend for our team#2018-06-2914:59mfikesWow! This will be extremely useful.#2018-06-2922:08cfleming@pesterhazy Great! I know a lot of people are waiting on it.#2018-06-3003:46dorabIf I try lein ancient after incorporating lein-tools-deps in my project.clj it always tells me that the artifacts are up-to-date. What do I need to do to get lein ancient work with lein-tools-deps? Thanks.#2018-06-3005:15hagmonk@dorab because with lein-tools-deps you're storing your deps in deps.edn, I'd recommend you use a tool that targets deps.edn directly. So far I only know of https://github.com/Olical/depot, but it works great#2018-06-3015:41dorabThanks!#2018-06-3015:48dorabDid exactly what I wanted.#2018-06-3018:04stathissiderishow alpha is add-lib? It fails for me like:
eggshell.controller> (deps/add-lib 'spec-provider {:mvn/version "0.4.14"})
true
eggshell.controller> (require 'spec-provider.provider)
FileNotFoundException Could not locate spec_provider/provider__init.class or spec_provider/provider.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name. clojure.lang.RT.load (RT.java:463)
#2018-06-3018:05stathissiderisIf I try to add-lib again, it returns false#2018-06-3018:05stathissideriswhich means that it didn’t have to add it again#2018-06-3018:11dominicm@stathissideris don't know why that's happening, buut, you need to be under a DynamicClassLoader for it to work.#2018-06-3018:11stathissiderisI’m not even sure what that means 🙂#2018-06-3018:27stathissiderisI’m in a REPL, is that ok?#2018-06-3018:27hiredmanadd-lib is very alpha#2018-06-3018:29stathissideris@hiredman sounds like I have to wait for a bit longer before I integrate it into my app#2018-06-3020:50dominicm@stathissideris it is possible to change the class loader of a repl, I believe it's in the recent history.#2018-06-3020:53alexmilleradd-lib works by finding the highest dynamic class loader in the current classloader chain and altering its loader sources #2018-06-3020:54alexmillerSome repl contexts do not have a shared dcl above all repl invocations#2018-06-3020:54alexmillerSo changing it would not necessarily have any effect, I think nrepl creates a new context on every evaluation#2018-06-3020:55alexmillerUsing clj repl works for me but nrepl things I’ve tried do not#2018-06-3020:56dominicmwe've been talking about how to fix this in nrepl.#2018-06-3020:57alexmillerJust need a dcl shared by all evals#2018-06-3020:58stathissideris@alexmiller I’m trying to integrate add-lib to my Swing app (and yes, I do realize it’s 2018 😄 ), the REPL entered the discussion because I naively thought that calling add-lib from the app would also make it available to the REPL I’m developing with#2018-06-3020:58alexmillerAre you using nrepl?#2018-06-3020:58stathissiderisyes#2018-06-3020:59stathissiderisbut having the new libs available in the repl is not essential to me#2018-06-3021:02stathissiderisI may need to read up on classloaders, I call add-lib from within a future, would that make a difference?#2018-06-3021:21stathissideris@alexmiller this is the classloader chain within the future:
(#object[sun.misc.Launcher$AppClassLoader 0x42a57993 "
Starting with (.getContextClassLoader (Thread/currentThread)). So no DCL there, I guess that explains it#2018-06-3021:22dominicmYou can do a set @stathissideris #2018-06-3021:23stathissiderishm?#2018-06-3021:23stathissiderisoh, setContextClassLoader!#2018-06-3021:25mfikes@stathissideris If it helps, I had to do something similar here https://github.com/mfikes/clojurescript/commit/d68c9397599366777d9b322ec586fdd398302f25#diff-2c521d7ba435fbef7d128f4295f86785R612#2018-06-3021:27stathissideris@mfikes thanks, that looks like it may work 🙂#2018-06-3021:37stathissideris@mfikes @dominicm some progress! it seems that setting the classloader as @mfikes demonstrated allows add-lib to work, but the added lib is still not available to the part of the app that needs to use it, presumably because it doesn’t share the same DCL… but now at least I understand how it works and I may be able to fix it somehow#2018-06-3021:38stathissiderismany thanks to all#2018-06-3021:39mfikes@stathissideris Cool. IIRC, in the change linked above, I shoved that DCL in at the earliest possible moment.#2018-06-3021:40mfikes(It is the very first bit of code executed in main, and my hope was that any threads spawned afterward would inherit it.)#2018-06-3021:42alexmillerthat should be the case#2018-06-3023:58cflemingJust to check my understanding of this - is it the case that alembic no longer works under nREPL, because of the dynapath change for Java 9? Sounds like nREPL used to actually modify the URLClassLoader but can no longer do so?#2018-07-0105:04cflemingActually, I’ll ask this over in #nrepl#2018-07-0103:11lincpa@stathissideris I encountered a similar failure during the use of Lein::
1. Add missing dependencies in project.clj (prompt in error message) 2.lein clean#2018-07-0200:39gfredericksthe +6 can be made less hacky using some sort of fancier "read up until you encounter this magic string" approach#2018-07-0200:41gfredericksstartup is like 7sec though O_O
I guess -Sdeps subverts caching?#2018-07-0212:56alexmillerIt used to but I think fixed in the current release#2018-07-0215:41gfredericksoooh#2018-07-0215:41gfredericksfaaancy#2018-07-0215:41gfredericksI will try it#2018-07-0215:43gfredericksyep, seems better#2018-07-0220:18martinklepschI’m running a script with clojure and have a println at the end. After that line is printed the script takes another 30+ seconds to exit. I’m probably just doing something weird but anyone an idea how to debug this?#2018-07-0220:19manutter51I don’t know about debugging, but that looks like the kind of problem that is often solved by adding a call to shutdown-agents somewhere at the end.#2018-07-0220:24martinklepsch@manutter51 that indeed resolved the issue, thanks!#2018-07-0308:31pesterhazy@martinklepsch probably not related to the println itself, right?#2018-07-0308:33martinklepschNo, println was just literal string, but the shot down agents thing fixed it for whatever reason :man-shrugging: #2018-07-0400:47hagmonkyou can try hitting it with jstack next time to see where it's hung up#2018-07-0917:28andy.fingerhutLikely there is something in your program that is creating a future, which causes that behavior, e.g. clojure.java.shell/sh -- http://clojuredocs.org/clojure.core/future#2018-07-1313:21martinklepsch@U0CMVHBL2 thanks that was most likely the issue. Is there any way to properly shutdown an individual future so that I don’t have to clean up using shutdown-agents later?#2018-07-1314:56martinklepschThis issue came up pretty frequently again, tried jstack now which returns the follow (besides lots of other stuff):
"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007f89a7016800 nid=0x5103 in Object.wait() [0x000070000957b000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c000e778> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000006c000e778> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007f89a7013800 nid=0x2e03 in Object.wait() [0x0000700009478000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c000e948> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x00000006c000e948> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
#2018-07-1315:01martinklepschduh, found it 🙂 didn’t stop a scheduler facepalm#2018-07-1315:04martinklepschSeems shutdown-agents is still required in some capacity though#2018-07-0522:03richiardiandreais there a known issue in clojure exit codes not forwarded with -m ?#2018-07-0523:01ghadiI haven't seen one @richiardiandrea can you make a reproduction case?#2018-07-0523:01richiardiandreabefore that I'll make sure it is not on my side 😄#2018-07-0611:48gfredericksdoes customizing a repl consist of having an alternate (different from clj) executable for starting the repl, or is there a facility in tools.deps to customize it? e.g., customize the printing function, or ensure that certain code is run before the repl starts#2018-07-0612:09alexmillerOnce you have a repl, you can start a nested repl within it with whatever customizations you want. But I think this an area ripe for better options.#2018-07-0612:26gfrederickslooks like I can't put a system-global user.clj file on the classpath automatically, is that correct?#2018-07-0612:27gfredericks(due to "only the last :paths is kept and others are dropped")#2018-07-0614:13ghadiyou can certainly start a different main method rather than clojure.main and give yourself a souped up repl @gfredericks#2018-07-0614:34gfredericks@ghadi but that requires passing args each time, right?#2018-07-0614:43ghadisi#2018-07-0614:43bronsasi`* :P#2018-07-0614:47ghadialias repl='java $(cat jopts) -cp $(clojure -Spath) clojure.main -m gfredericks.fave.repl'#2018-07-0707:13cflemingAs I understand it, aliases are never transitive. So this means that when you depend on a project using, say, :local/root then you will only ever see deps/paths from your dependency which appear in the top level of the deps.edn file, correct?#2018-07-0707:14dominicm@cfleming No transitive aliases, correct.#2018-07-0721:58martinklepschHey folks 👋 i have two questions: a) (how) does tools.deps support maven scopes? & b) is there a way to include dependencies with “provided” scope in the classpath? Thanks! 🙏#2018-07-0722:15alexmillera) not supported and b) that’s not a thing you should do anyways#2018-07-0810:26cflemingb) is absolutely useful. I use it to compile Clojure code against the IntelliJ classes, but not have them included when I run.#2018-07-0810:27cflemingWhat would be the deps solution to that - two separate classpaths using aliases? I guess I’d need an alias for the IntelliJ classes and would have to use that when compiling but not running?#2018-07-0810:53dominicmaliases is how I've solved it up until now.#2018-07-0815:22alexmillerYes#2018-07-0815:24alexmillerYou seem to be describing using provided scope aliases in your own project (which is useful, but handled by aliases). The question I read above though sounded like how to include transitive provided-scope deps, which does not seem like a thing you should do#2018-07-0816:38martinklepschYeah, I need b) for a similar tooling related concern #2018-07-0816:53martinklepschI want to load code from Artifact A which has a “provided” dependency B. Now I’d like to make sure all these dependencies are loaded without manually listing those “provided” deps. Hope that makes sense#2018-07-0816:56alexmillerWell, the whole point of provided scope is: you provide them#2018-07-0816:57alexmillerSo, not going to make something to do this#2018-07-0817:31dorabIs resolve-deps supposed to download the relevant dependencies if I call it myself?#2018-07-0817:31dorab{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "c0a4afb2a1a280a4afb2a1a2ed8fb0b4a990aca5b8edf9f0f2f0"}, :content ("[email protected]")}#2018-07-0817:36dorabIf it is supposed to work, what am I doing wrong above?#2018-07-0817:37dorabBTW, this is in a directory (my home dir) that does not have a deps.edn in it.#2018-07-0821:24alexmillerYou need to provide the :mvn/repos to look in (these are in the install deps.edn with clj)#2018-07-0821:53dorabAh. I thought those would automatically be included. So, I should create the full deps.edn map myself and then pass that to resolve-deps. Thanks.#2018-07-0901:40dnolenseems like there’s a bug with git deps where top-level :paths applies instead of the defaults or whatever is supplied in the git deps.edn#2018-07-0903:22alexmillerYes, there is a ticket for it#2018-07-0911:07dnolenah k#2018-07-0913:04dominicmWorse than the silently ignoring problem, I seem to have encountered a case where all aliases are ignored if one of them is not present.#2018-07-0913:06alexmillerI’m planning to look at this today#2018-07-0913:12dominicmfwiw, they weren't all ignored. Just no fetching took place. I haven't looked hard at this, but it confused me.#2018-07-0920:40kennyI seem to be running into an issue with local deps. If I have this directory structure
app
lib1
lib2
with app/deps.edn as
{:deps {lib2 {:local/root "lib2"}}}
and lib1/deps.edn as
{:deps {}}
and lib2/deps.edn as
{:deps {lib1 {:local/root "../lib1"}}}
and I run clj -Spath in the app directory, I get this message:
Error building classpath. Manifest type not detected when finding deps for lib2/lib2 in coordinate #:local{:root "lib2"}
Are nested local dependencies supposed to work?#2018-07-0922:24alexmillerShouldn’t your app one have ../lib2 ?#2018-07-0922:47kennyYou're absolutely right. That's what I get for working while hungry...#2018-07-0922:51hagmonkOh wow, I came here to report the same thing, except I've had lunch so I know it's a bug :)#2018-07-0922:53kennyI think I'm hitting the same problem with a slightly different directory structure. Except this time it is correct.
lib1
lib2
samples/sample1
with lib1/deps.edn as
{:deps {}}
and lib2/deps.edn as
{:deps {lib1 {:local/root "../lib1"}}}
and samples/sample1/deps.edn as
{:deps {lib2 {:local/root "../../lib2"}}}
and I run clj -Spath in the samples/sample1 directory, I get this message:
Error building classpath. Manifest type not detected when finding deps for lib1/lib1 in coordinate #:local{:root "../lib1"}
#2018-07-0922:53hagmonkYeah same issue:
./libs/dep-b
{:deps {org.clojure/clojure {:mvn/version "1.9.0"}}}
./libs/dep-a
{:deps {dep-b {:local/root "../dep-b"}}}
./apps/foo-app
{:deps {dep-a {:local/root "../../libs/dep-a"}}}
#2018-07-0922:55hagmonkI can work around it by having libs/dep-a refer to ../../libs/dep-b instead, but that is only a coincidence in this case, if apps was one level deeper it wouldn't work#2018-07-0922:57hagmonklooks like it's already reported here: https://dev.clojure.org/jira/browse/TDEPS-74#2018-07-0922:59kennyOh ok. That's a nasty one.#2018-07-0922:59hagmonkpush the "vote" button :)#2018-07-0923:03kennyIt looks like the user in that issue is able to get output from clj -Spath though.#2018-07-0923:03kennyBut it sounds identical. Perhaps an older version of tools-deps didn't have this error message.#2018-07-0923:07hagmonkI thought :deps/root might be another way to dig oneself out of this hole, but perhaps I'm misunderstanding how that works#2018-07-1012:48pesterhazyI know that you can specify local exclusion in deps.edn like so "reagent {:mvn/version "0.7.0", :exclusions [cljsjs/react-dom cljsjs/react cljsjs/create-react-class cljsjs/react-dom-server]}" Is there a way to specify global exclusions as well?#2018-07-1014:41alexmillerNo#2018-07-1204:18cflemingI’m working on some tools.deps testing, and I can’t figure this out. I’m calling resolve-deps like this:
(resolve-deps
{:deps {'local {:local/root "/Users/colin/dev/tools-deps-test/library"}}}
nil)
#2018-07-1204:18cflemingThat deps.edn looks like this:
~/d/cursive (deps)> cat /Users/colin/dev/tools-deps-test/library/deps.edn
{:deps {org.clojure/tools.gitlibs {:git/url ""
:sha "cab6abf0122cffb9e6b947c838458c69a0992031"}}}
#2018-07-1204:19cflemingWhat I’m getting back in my lib map includes this:
local {:local/root "/Users/colin/dev/tools-deps-test/library",
:deps/manifest :deps,
:deps/root "/Users/colin/dev/tools-deps-test/library",
:paths []},
#2018-07-1204:19cflemingI can’t figure out why the :paths vector is empty here.#2018-07-1204:19cfleming/Users/colin/dev/tools-deps-test/library/src exists.#2018-07-1204:41cflemingUgh, never mind me, I’m not merging in my config maps.#2018-07-1204:41cflemingThis works:
(let [deps (reader/merge-deps [(reader/slurp-deps "/usr/local/Cellar/clojure/1.9.0.381/deps.edn")
(reader/slurp-deps "/Users/colin/.clojure/deps.edn")
(-> {:deps {'local {:local/root "/Users/colin/dev/tools-deps-test/library"
:deps/manifest :deps}}}
(#'reader/canonicalize-all-syms))])]
(resolve-deps deps nil))
#2018-07-1215:38kennyIt seems tools-deps suffixes all Git deps with each path in my :paths vector. If I have ["src" "foo1" "foo2"] for my :paths, the output of clj -Spath will suffix each Git dependency with each path. i.e. /Users/kenny/.gitlibs/libs/lib1/lib1/f31e19ffe371a60f0474759451dba9b88bf7ef04/src:/Users/kenny/.gitlibs/libs/lib1/lib1/f31e19ffe371a60f0474759451dba9b88bf7ef04/foo1:/Users/kenny/.gitlibs/libs/lib1/lib1/f31e19ffe371a60f0474759451dba9b88bf7ef04/foo2. Is this intentional?#2018-07-1215:39dominicmknown bug#2018-07-1215:40kennyIs there an issue?#2018-07-1215:42dominicmThere is, although I'm not sure of the ref for it#2018-07-1215:48kennyI believe it's https://dev.clojure.org/jira/browse/TDEPS-52#2018-07-1219:24aviI got bit by that yesterday, working with a local lib. My app has main test-runner etc under src but the lib doesn’t have main so none of the clj files could be found by require/load#2018-07-1219:25aviAh no, my problem was slightly different#2018-07-1219:25aviAlmost the inverse#2018-07-1219:25avi(Just read the issue more closely)#2018-07-1219:25aviI’ll try to check whether an issue for my experience already exists.#2018-07-1302:22dnolenare :extra-deps actually merged? like if in an alias I want :exclusions but not otherwise?#2018-07-1302:32alexmillerI’m not sure I understand what you’re asking but it sounds like :override-deps#2018-07-1302:33dnolenoh! cool 🙂#2018-07-1302:35dnolenthanks#2018-07-1312:21martinklepschNot sure if a bug but clj -Sdeps '{:deps {org.asciidoctor/asciidoctorj {:mvn/version "1.5.6"}}}' -Spath doesn’t pull in the dependencies below (lein/boot do)
[org.asciidoctor/asciidoctorj "1.5.6"]
├── [com.beust/jcommander "1.35" :scope "runtime"]
└── [org.jruby/jruby-complete "1.7.26" :scope "runtime"]
#2018-07-1312:26dominicmRuntime scope dependencies aren't or weren't brought in. I think they are now. #2018-07-1611:45martinklepschDo I need to manually upgrade tools-deps for that to take effect somehow?#2018-07-1611:45martinklepschsorry noob q 😄#2018-07-1612:08dominicmhttps://dev.clojure.org/jira/browse/TDEPS-26#2018-07-1612:10dominicmI guess you bump your version? Not sure when the last release was#2018-07-1321:35dnolenI guess there’s no error on non-existent alias?#2018-07-1321:36alexmillerHa, I just committed that within the last hour :)#2018-07-1321:36alexmillerWill be in next release#2018-07-1322:23rbergerIf one’s ssh key for github is not id_rsa or any of the default ssh names, do you need to specify the name somewhere in deps.edn? I can git clone a private dependency repo with my keys in the ssh-agent, but the clojure command always ends up with:#2018-07-1322:23rbergerError building classpath.
#2018-07-1322:41rbergerThis is on macos High Sierra and Clojure 1.9.0.381#2018-07-1323:29ghadiThis is an awful answer, and it's a bit of a mess, and I'd really like to fix it once and for all by shelling out to git directly (golang does this):
@rberger
for now, make sure the environment SSH_AUTH_SOCK is set to your ssh-agent's socket path
move any extraneous Identity keys in your .ssh/config out of the way#2018-07-1323:29ghadiJGit / JSch tries to read them and gets confused#2018-07-1323:29ghadi(I use ed25519 keys, which are common these days...only supported via the ssh agent)#2018-07-1323:39rbergerThanks! It looks like I had some things in the ~/.ssh/config that was causing the problem. Might be good to at least mention this in the docs (https://clojure.org/reference/deps_and_cli)#2018-07-1323:41rbergerThough the info in the config file is the correct path to the IdentifyFile.
Host
User git
Port 22
HostName
IdentityFile ~/.ssh/github.rsa
AddKeysToAgent yes
TCPKeepAlive yes
IdentitiesOnly yes
#2018-07-1514:01alvinathere is command line tool for finding latest version of the software#2018-07-1514:01alvinacan some body remind me the name of that command?#2018-07-1516:28dorabWill https://github.com/Olical/depot work for you?#2018-07-1520:11pesterhazy@rumaysaalvina2 @hagmonk's https://github.com/hagmonk/find-deps#2018-07-1600:37alvinathanks a lot#2018-07-1608:14levitanongHi all, trying to get the clojure:tools-deps docker image to work with gitlab-ci. Does anyone have experience in the matter?#2018-07-1609:16Andreas LiljeqvistI have never used gitlab-ci, but here is my .yml for running on bitbucket.
I imagine it would look quite similar.
https://pastebin.com/SE1shJE3#2018-07-1610:42delaguardoWhat kind of problems you are seeing?#2018-07-1611:07levitanong@U04V4KLKC clj: command not found. Same with clojure. I’ve tried purposefully misspelling the docker image to see if there’s a distinct error when the docker image isn’t found, but it appears gitlab-ci just silently fails, and proceeds to run as if from a blank image.#2018-07-1611:08levitanong@U7YG6TEKW Yeah, mine looks similar, except gitlab complains that the clojure command isn’t found.#2018-07-1611:15delaguardoclj should is not available cause readline is not installed. But clojure should be there.#2018-07-1611:16delaguardoCould you show your config file?#2018-07-1611:21delaguardoAlso I can suggest to use clojure:tools-deps as your image so it is hard to misspell the name#2018-07-1611:28levitanongtest:tools-deps:
image: clojure:tools-deps-alpine
script:
- clojure -A:test
#2018-07-1611:29levitanongSkipping Git submodules setup
$ clojure -A:test
bash: line 52: clojure: command not found
ERROR: Job failed: exit status 1
#2018-07-1611:33delaguardoThat is odd… My config file looks a bit different, but it should not be a problem#2018-07-1611:34levitanongOne of the first things I tried was:
image: clojure:tools-deps
test:
script:
- clojure -A:test
#2018-07-1611:34levitanongsame error 😮#2018-07-1611:37delaguardoWhich version of gitlab-runner you are using?#2018-07-1611:38levitanongRunning with gitlab-ci-multi-runner 9.5.1 (96b34cc)#2018-07-1611:41delaguardoI’m using 11.0. Let me downgrade, it can be a problem#2018-07-1611:43levitanongooh#2018-07-1612:00delaguardoUnfortunately, I can not downgrade to 9.5.1 on my mac. Looks like a bunch of dependencies conflicts there. Is it possible for you to upgrade yours and try again?#2018-07-1612:02levitanongHmm. I’m working off of my company’s on-prem installation of gitlab. I’ll ask the dev-ops guy if he can upgrade it. I’ll keep you posted!#2018-07-1614:12levitanongApparently our devops guy needs to backup all our repos over the weekend to upgrade gitlab. I’ll prolly have to get back to you in a week. Good thing lein still works!#2018-07-2403:43levitanonghello, tried again on gitlab-runner 11.1.0. Still says clojure: command not found. Do you think it’s because we’re using the on-prem community version of gitlab?#2018-07-2406:28levitanongOop, scratch that. we had a faulty config somewhere else. Works now, thanks!#2018-07-2406:44delaguardoNice! parrot #2018-07-1609:58darnokHi. I have a problem w tools-deps. I'm using :local/root quite extensively and I noticed a problem. If I have a project project1 that points to another library with {:local/root "../../libs/abc"} and then that library uses another library {:local/root "../other-lib" it doesn't resolve the abc library classpath correctly.#2018-07-1610:00darnokIt resolves the second local/root according to the project1 root.#2018-07-1610:44dominicmyeah. There is a JIRA for that.#2018-07-1610:45dominicmI've been able to "flatten" everything up until now though.#2018-07-1611:31darnokI think it should be fixed https://dev.clojure.org/jira/browse/TDEPS-74. Probably, it's not that easy to fix.#2018-07-1611:45martinklepsch# clj
Please install rlwrap for command editing or use "clojure" instead.
# echo $?
0
has changing the status code here to some non-zero value been discussed before?#2018-07-1612:08delaguardoI remember such discussion but can’t find an issue. But I’m definitely want to vote for it#2018-07-1612:44alexmillerno ticket, but feel free to log on and/or provide a patch#2018-07-1613:44martinklepsch@U064X3EF3 can you point me to the source of the clj clojure scripts maybe? thanks 🙂#2018-07-1613:45alexmillerTickets go to https://dev.clojure.org/jira/browse/TDEPS, patches for that stuff will be on https://github.com/clojure/brew-install (scripts are at src/main/resources)#2018-07-1613:55martinklepsch@U064X3EF3 https://dev.clojure.org/jira/browse/TDEPS-86#2018-07-1612:04stathissiderishello, is there some special magic to make git deps from private repos work? I’m using the {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "ddbab4a99dbab4a9b5a8bff3beb2b0"}, :content ("[email protected]")} URL and auth fails#2018-07-1612:06delaguardoYesterday was discussion about it - https://clojurians.slack.com/archives/C6QH853H8/p1531520620000066#2018-07-1612:07stathissideristhanks! I’ll have a look#2018-07-1713:16martinklepschIs there a way to specify an entirely different set of :deps for an alias? (i.e. no merge, no override, just different value)#2018-07-1713:21dominicm@martinklepsch in what context?#2018-07-1713:23martinklepsch@dominicm not sure I understand what you mean by context? 😄#2018-07-1713:32martinklepschI’d like to swap out the value of :deps in the top level map by another value via an alias#2018-07-1713:54dominicmSo from the clj command line use? What's the use case?#2018-07-1714:11ghadi:override-deps should do it, but I think there is an acknowledged bug when the override is of a different "type" (mvn/git)#2018-07-1714:24ghadihttps://dev.clojure.org/jira/browse/TDEPS-51#2018-07-1714:58dominicm@ghadi I think @martinklepsch means that he wants to have:
{:deps {A {:mvn/version "1.1.0"}}
:aliases {:B {:the-new-deps {B {:mvn/version "5.4.1"}}}}}
And when doing clj -A:B that should result in only B being on the classpath#2018-07-1714:59ghadioh#2018-07-1714:59martinklepschyeah, that’s it @dominicm#2018-07-1715:00ghadiI think this has come up before -- not supported by aliases, but perhaps by a different deps.edn file#2018-07-1715:00dominicmI don't understand why you'd practically need that though 🙂#2018-07-1715:00dominicmI suppose you could actually have:
{:deps {}
:aliases {:A {:extra-deps {A …}} :B {:extra-deps {B …}}}}
#2018-07-1715:01martinklepschyeah aliases would work I think#2018-07-1715:02martinklepschI’m not 100% sure I need it either but I come up with some approaches and have no idea what tools-deps allows for so just asking 😄 sorry for the noise 🙂#2018-07-1715:13ghadino problem @martinklepsch not noise at all#2018-07-1715:13ghadiIt does seem a bit "off-label prescription"#2018-07-1715:13ghadi(the use case)#2018-07-1715:31martinklepsch@ghadi cool, thanks 🙂#2018-07-1715:32martinklepschIf I specify a deps.edn with -Sdeps any local deps.edn in the working directory should be ignored — is that correct?#2018-07-1715:32dominicmNo, though others have tripped over that wording too.#2018-07-1715:33martinklepschuh, yeah, especially that “final” part: “Deps data to use as the final deps file”#2018-07-1715:37martinklepschis there another way to achieve it then? specifying a file would also be fine, just looking for something that ignores an existing deps.edn#2018-07-1716:53dominicmFinal in the context of merging together deps.edn files 🙂#2018-07-1717:45martinklepschHm, the word “final” creates a different expectation for me 😄
So is the only way to run a clj process with a deps.edn that’s different than the one in the current working directory to move files around?#2018-07-1718:01dominicmI think the proposal was to change it to last.#2018-07-1718:02dominicmNot that I know of. I think pack is unique in utilizing tools.deps.alpha in that way, and I'm not sure it really works due to the cwd being internal to t.d.a rather than passed in. #2018-07-1718:11martinklepschMaybe another option could be added#2018-07-1718:20martinklepschhttps://dev.clojure.org/jira/browse/TDEPS-87#2018-07-1718:57alexmiller-Sdeps!#2018-07-1719:23dominicmHow well does that work in shells? #2018-07-1722:38alexmillerI was actually kidding :)#2018-07-1805:14dominicmVim wasn't (for its internal commands). #2018-07-1808:13dominicmhttps://github.com/nrepl/nREPL/issues/8 ah, that explains why add-lib doesn't work with nrepl. I was hoping to patch nrepl, but it seems harder now.#2018-07-1808:17dominicmOh actually, when add-lib refers to the "highest" class loader, I guess that means it would find the parent one if I set it, so I could fix this in nrepl#2018-07-1809:53dominicmhttps://github.com/nrepl/nREPL/pull/35 resolved! (pr merged)#2018-07-1808:40martinklepschUnder what conditions is it expected that a user has to manually delete .cpcache?#2018-07-1809:54dominicmbugs? 😄 What happened?#2018-07-1809:54dominicmI wouldn't expect to ever have to from what I know, fixes are being made to handle cases where it isn't being updated correctly#2018-07-1810:17pesterhazy@U050TNB9F I think this is an interesting question#2018-07-1812:24alexmillerGenerally using -Sforce to force recomputing a single classpath makes more sense. One known case where this is required is when local transitive deps change their dep structure - right now that won’t be automatically discovered#2018-07-1813:00martinklepsch> local transitive deps change their dep structure
that might have been it then#2018-07-1814:46alexmillerI have a plan to fix this, but it’s kind of involved#2018-07-1810:22martinklepschWith git-libs is there a way to specify another “root” path to look for a deps.edn in?#2018-07-1810:22martinklepschI remember this being discussed previously but can’t recall#2018-07-1810:22dominicmYep! I think :deps/root is the key#2018-07-1810:50martinklepschError building classpath. Manifest type not detected when finding deps for codox/codox in coordinate {:git/url "", :sha "ac6c83a4c7d334bbc27a1bad4240ed7e5fa04ff2", :deps/root "codox/"}
Hrm, I must be missing something. For some reason manifest here seems to be nil: https://github.com/clojure/tools.deps.alpha/blob/84826fef06c725dac0c3d63cfbfd0916553b7cf9/src/main/clojure/clojure/tools/deps/alpha/extensions/git.clj#L45-L47#2018-07-1810:51dominicmdo you have a deps.edn file?#2018-07-1810:51dominicmor pom.xml#2018-07-1810:51cflemingDoesn’t :deps/manifest only support :deps at the moment?#2018-07-1810:52cflemingi.e. deps only knows how to extract path info from other deps projects?#2018-07-1810:53dominicmI think there's a :deps/manifest :mvn implementation#2018-07-1810:55martinklepschduh, I thought I commited the deps.edn file but didn’t 🙈#2018-07-1822:49zentropeI’ve got a github repo with mname/foo with a clojure project inside it, mname/foo/clojure-project/: is it possible to use that for a deps.edn git dependency?#2018-07-1822:50zentropeAnswer seems to be “no” as I can’t reference a github URL + subdir.#2018-07-1822:51zentropeOh, is that the :deps/root thing I see up above there? Hm.#2018-07-1822:55zentropeGetting a “don’t know how to create ISeq” from the -m namespace part of the CLI, but at least it seems to find the right subdirectory.#2018-07-1822:55zentropeOops, typo. Nevermind.#2018-07-1822:55zentrope:+1:#2018-07-1903:24cflemingCursive support for deps is out!: https://groups.google.com/d/topic/cursive/32tfBegzfVc/discussion#2018-07-1903:25cflemingStill lots to do and lots of potential changes.#2018-07-1908:31claudiuHi, is there any tools-deps alternative for https://github.com/luminus-framework/lein-uberwar ?#2018-07-1908:32dominicm@claudiu I'd be interested in adding uberwar support to https://github.com/juxt/pack.alpha I'm not familiar with the format really though and what would be needed. Is it just a jar with a particular manifest?#2018-07-1909:11claudiunot really sure to be honest. 🙂 I am deploying my test-app to google-appengine, and that's the format there.#2018-07-1909:12claudiuAdded lein-tools-deps to my project, seems to work for development but the uberwar seems to compile multiple times the same file, and when I run it I get some strange errors.#2018-07-1909:19dominicmI can try find some time to look into it for you.#2018-07-1909:23dominicmLooks like a jar with a specific fs layout, shouldn't be too difficult to get working with pack, I'm a little worried about git dependencies, that might need some testing.#2018-07-1910:13claudiuCool 🙂 Could help with testing git & normal, and see how it matches the lein one.#2018-07-1911:24claudiuHaving the uberwar option would be really awesome.#2018-07-1917:28dominicmIs there an easy way to test an uberwar locally?#2018-07-2009:15claudiuI have only used uberwar in appengine context. The simplest way to test the war I can think of is : https://gist.github.com/claudiuapetrei/540c8aca1ef53d86f75f123b319a3703#2018-07-2010:06dominicmThanks, I'll take a look#2018-07-1915:44alexmillerreleased tools.deps.alpha 0.5.442, clj 1.9.0.391#2018-07-1915:45alexmiller- Throw error if unknown aliases are used (TDEPS-85)
- Fix bad break character in rlwrap (TDEPS-77)
- Use non-0 exit code in clj if rlwrap doesn't exist (TDEPS-86 - thanks Martin Klepsch!)
- Change wording for -Sdeps in help and man for clarity (TDEPS-87)
#2018-07-2000:09cflemingI’m testing deps with Cursive, and I’m finding that resolving several large projects at once is really slow. This is a project with several apps in it, all of which use a series of libraries transitively via ‘:local/root’. Is it possible that each project refresh works out a complete list of what to download up front and then downloads it, such that each project will download everything even if another project has concurrently downloaded that dep in the meantime?#2018-07-2000:14cflemingIn total there are 15 projects (including apps + libs), it uses kafka so who knows how much each one will download. But I’m on a 1 Gb fibre connection, so it’s generally pretty quick. It’s taking around 15-20 minutes to resolve them all.#2018-07-2000:32alexmillerIt uses Maven to download as it goes, which uses the Maven cache#2018-07-2000:33alexmillerThat said, I have spent no time optimizing any of that and none of it is parallel (but most likely some could be)#2018-07-2000:34cflemingRight, my case is probably unusual in that if you set up a large project in Cursive (which I just did based on data a user sent me) then IntelliJ will go off and resolve all the projects in parallel.#2018-07-2000:40cflemingDoes the maven lib separate the resolution into a resolution step and a download step? I’m wondering if each project does the resolution step, decides what to download and then downloads it without checking before each download if something else has done so in the meantime?#2018-07-2012:59mpingHi, is there a recommended way to integrate lein with tools-deps? I want to use git dependencies#2018-07-2013:09andre.stylianosThe only one I know: https://github.com/RickMoynihan/lein-tools-deps#2018-07-2013:09mgrbyte@mping Not sure about "recommended", but I'm using lein-tools-deps - works pretty well so far. Only things that don't work are lein deps :tree, lein search AFAICT#2018-07-2013:09mping@mgrbyte :+1:#2018-07-2015:31hlolliis there a function in tools.deps that I could use to generate a vector of maps containing all remote dependencies for a project's dependencies, plus, if possible, the sha-sums of the .jar files. I'm wiring a deriviation for nix(Os), and it seems that there's no tool for generating nix expression for clojure dependencies available. There's one for maven, called mvn2nix. I guess I could use that, but I'd have to make some hack for mirrors, adding clojars etc..#2018-07-2015:32dominicmyeah, I think that would be possible via the api#2018-07-2015:33dominicmthe api is small, call both of the documented functions#2018-07-2015:33hlollimake-classpath and resolve-deps? I will try!#2018-07-2015:35dominicmmaybe not make-classpath actually#2018-07-2015:35dominicmjust resolve-deps & vals#2018-07-2015:53rickmoynihan@alexmiller: should the tools.deps library be including slf4j-nop?
https://github.com/clojure/tools.deps.alpha/blob/744c9c027b1b0d71081a7903b517abbd960a9849/deps.edn#L15#2018-07-2016:49hlolli@dominicm thanks for the tip, I'm getting all the deps printed, which is good. But that which is missing, is the url's to the .jar files
fipp
{:versions
{#:mvn{:version "0.6.12"}
{:mvn/version "0.6.12", :deps/manifest :mvn}},
:paths {#:mvn{:version "0.6.12"} #{[]}},
:select #:mvn{:version "0.6.12"},
:pin true},
probably straightforward to do, but how would I get the download urls from this information?#2018-07-2016:51dominicm@hlolli I think there's something missing here, you've called resolve-deps?#2018-07-2016:51hlolliyes I did this
(prn
(pprint (resolve-deps '{:deps
{org.clojure/clojure {:mvn/version "1.9.0"}
org.clojure/clojurescript {:mvn/version "1.10.238"}
org.clojure/tools.reader {:mvn/version "1.3.0-alpha3"}
com.cognitect/transit-cljs {:mvn/version "0.8.256"}
malabarba/lazy-map {:mvn/version "1.3"}
fipp {:mvn/version "0.6.12"}}}
{:verbose true}
)))
#2018-07-2016:55dominicmSomething looks off about your result to me. The value of :paths for me is different, I have jars in there.#2018-07-2016:55dominicmor dirs, context dependent.#2018-07-2016:57hlolliah yes, if the jar is found, it appears#2018-07-2016:57hlollibut only locally#2018-07-2016:57hlollicom.google.protobuf/protobuf-java
{:mvn/version "3.0.2",
:deps/manifest :mvn,
:paths
["/home/hlolli/.m2/repository/com/google/protobuf/protobuf-java/3.0.2/protobuf-java-3.0.2.jar"],
:dependents [com.google.javascript/closure-compiler-unshaded]},
#2018-07-2017:01dominicmYou want the remote values?#2018-07-2017:03hlolliyes, I need that, and the sha sums, but I could just get them after all is downloaded#2018-07-2017:07hlollithe pain with nix, is that nothing can be downloaded without being declared, all http fetches need to be done on nix's side#2018-07-2017:08dominicmI don't think you can get the remote urls, I think it's blackboxed into a maven library#2018-07-2017:09hlollimaybe, you can always download jars directly from clojars, maybe maven is more blackboxed#2018-07-2017:10dominicmI meant maven library which handles clojars & maven#2018-07-2017:10dominicmt.d.a has no concept of "get-url-for-dep"#2018-07-2017:10dominicmYou'd probably have to utilize the same approach that mvn2nix uses, and also write equivalents for git & local deps.#2018-07-2017:12hlollipff, hope its easy 🙂 will research.#2018-07-2022:31noprompthas anyone run in to problems with using krypton and git deps?#2018-07-2022:32nopromptclj complains it can’t find the ~/.ssh/id_krypton private key which it shouldn’t because, as i understand it, krypton has that key.#2018-07-2105:32dominicm@noprompt I imagine jgit doesn't support krypton#2018-07-2109:39dominicm@alexmiller @ghadi curious whether you've looked into libgit and whether it could be used in place of jgit to solve this problem? Maybe before falling back on jgit on any unsupported platforms?#2018-07-2109:40dominicm> Linux, FreeBSD, OpenBSD, Mac OS X, iOS, Amiga, MinGW and fully native Windows.
is pretty good though, "Amiga", like, come on!#2018-07-2109:40dominicmThere's a library, "jagged" for doing libgit w/ java, but it's not on maven.#2018-07-2113:32pesterhazyif it's fast enough, shelling out to /usr/bin/git seems optimal#2018-07-2113:36dominicmHaving tried to wrap libgit2 this morning, I'm thinking that shelling out is increasingly valuable. I'm wondering about environments which may not have the git binary available though.#2018-07-2113:36dominicmHaving libgit2 embedded for that situation is very valuable.#2018-07-2113:37dominicmOf course, /usr/bin/git falling back onto jgit or something would work for that pretty well#2018-07-2113:48pesterhazytrue, I didn't think of the case where the binary is not available (not unrealistic in this world of Docker)#2018-07-2118:10athomasoriginalDoes anyone know of examples of clj cli tool + deps.edn monorepo setups? I have been studying Juxt's Edge repo, but this does not yet include any other modules. My assumption is that one would create a :dev alias where, instead of referencing the artifact, it references the local path to your module...curious as to how other's approach this 🤔#2018-07-2119:12ghadiWe need to shell out to git to solve all the ssh related issues#2018-07-2119:13ghadiAll the other git based deps tools do that, like golangs#2018-07-2119:24athomasoriginal@ghadi sorry, was this in reference to my question?#2018-07-2119:25ghadiSorry, earlier Q about libgit#2018-07-2119:40dominicm@ghadi libgit doesn't do the same thing as git? #2018-07-2120:59cflemingIntelliJ also shells out to Git for its support.#2018-07-2120:59cflemingThere’s probably a bunch of useful support code there if we need it (Apache licensed)#2018-07-2219:13uwoare :exclusions only for mvn deps, or do they work for git deps as well? Running clj -Stree made me think they were, because the exclusion list appeared to be applied to the dep tree#2018-07-2220:21alexmillerShould work for both#2018-07-2221:30uwoawesome thanks!#2018-07-2220:20hlolliSo I succeeded with using tools.deps for generating nix-expressions, https://github.com/hlolli/clj2nix now it should be easy for nix users to start bombarding clojure-based application into nixpkgs, or use nix to build/deploy clojure applications. If you're a nix user, please give it a try 🙂#2018-07-2221:57dominicm@hlolli how did you do it? #2018-07-2222:02hlolli@dominicm after looking at a nix function fetchMaven, I realized that all maven2 remote url's follow a pattern, from there I discovered fetchMavenArtifact, which I use, for both clojar and maven central. Just try both for each dep, and both clojar and maven central work with same url format.#2018-07-2222:03dominicm@hlolli do you handle git and local? #2018-07-2222:08hlolligood question. I don't handle it automatically. And it's possible to do it.#2018-07-2222:09hlolliAnother thing I need to add is classifier. Which I don't handle, or the funtcion fetchMavenArtifact doesn't handle it.#2018-07-2222:20dominicmI'm not certain if t.d.a handles classifier yet, I might be wrong#2018-07-2222:20hlolliI'm about to test it 🙂, curious.#2018-07-2222:57alexmillerIt does#2018-07-2222:59alexmillerWith the caveat that classifiers do not distinguish in a lib (they’re part of the coord, not the lib key), so you can’t use multiple classifiers (or no classifier) for the same lib at the same time#2018-07-2223:00alexmillerIt also only handles jars and ignores other extension types#2018-07-2310:42cflemingIs there a rough ETA on Windows/Powershell support?#2018-07-2314:21alexmillerSorry, I just haven’t had time to pull it over the line yet. I expect it will be a couple weeks before I get back to it#2018-07-2321:14gfredericksI'm thinking about how to best replace all the tools that in leiningen mostly consisted of :injections, without creating a combinatorial explosion of mains
I had the idea that maybe that could be what a single user.clj could be for#2018-07-2321:15gfredericksI'm assuming tools.deps lets you add common source paths in the user's deps.edn, but I might have made that up#2018-07-2400:16gfredericksit works, at least:
:paths ["/home/memyself/.clojure/user-classpath"]#2018-07-2415:15timgilbertBest of luck with the Windows stuff. I've recently been running into this fiendish error with lein: https://stackoverflow.com/questions/201816/how-to-set-a-long-java-classpath-in-windows/201969#2018-07-2423:38souenzzoreal funny the kind of problem and solutions that windows users do in 2018.
WLS still not a thing?#2018-07-2513:37timgilbertWSL work pretty well, and you can run the ubuntu version of tools.deps on it with no problems. The pain comes when you are trying to interoperate with native Windows stuff - eg, IntelliJ runs natively, so it has its own notion of an m2 repository, classpath, etc#2018-07-2416:04ghadi@argument files
one or more argument files containing options
Java now accepts a file to read arguments ^#2018-07-2423:23gfredericksthe worst part is having to hard-code my home directory in the deps.edn; I'm not sure if there's a way to avoid that#2018-07-2500:25aviRelative paths? Paths that include things like ~ ? (Not sure what kind of thing ~ is. Alias?)#2018-07-2500:42gfredericksit's a bashism I believe#2018-07-2500:42gfredericks. and .. are a lot more first-class than ~#2018-07-2500:42gfredericksI can't use ~ because it's just bash, and I can't use . and .. because I assume that'd refer to the working directory of the java process, which could be anywhere#2018-07-2519:38pesterhazyAn equivalent of expanduser is easy to implement: https://stackoverflow.com/questions/6803913/java-analogous-to-python-os-path-expanduser-os-path-expandvars but not sure it’s a good idea#2018-07-2711:02gfredericksbut this is in a deps.edn file, where, unless I'm mistaken, there's no opportunity to execute code#2018-07-2809:32pesterhazyright, I meant as a potential feature for tools.deps. Many tools parsing yaml/json/ini configuration files process filenames with expanduser#2018-07-2812:35gfredericksah right#2018-07-2613:31mitchelkuijpersIs anyone here using this on circleci? I think it is a good idea to cache the .cpcache folder? Because I saw this in the reference guide:
The cached classpath file is used when:
* It exists
* It is newer than all deps.edn files
I would guess if i checkout the project and then restore a cache the .cpcache folder is always newer#2018-07-2613:31mitchelkuijpersHmm this is exactly what you want if I base the cache on deps.edn.. While I typed this I figured it out lol#2018-07-2614:04mitchelkuijpersIs there some place where it caches git dependencies? I am currently caching the .m2 folder and the .cpcache folder#2018-07-2614:26mfikes@mitchelkuijpers by default ~/.gitlibs#2018-07-2614:28mitchelkuijpersAh nice thnx#2018-07-2619:07Peter Wilkinshi, anyone know how to add a local maven repository to a deps.edn file?#2018-07-2619:11mfikes:mvn/local-repo#2018-07-2619:13mfikesSee https://github.com/clojure/tools.deps.alpha/blob/4f24e4ea6911e9a30a14979386f2b2cc3ad2ecc8/src/main/clojure/clojure/tools/deps/alpha/extensions/maven.clj#L31-L35#2018-07-2619:16mfikesSo, for example, this will create and stash stuff in ./my-local-repo
clj -Sdeps '{:mvn/local-repo "my-local-repo"}'
#2018-07-2711:09gfredericksis add-lib likely to get officialized anytime soon?#2018-07-2714:25alexmillerDefine soon#2018-07-2715:09mikerodI’m seeing 2 definitions:
1) in or after a short time.
2) used to indicate one’s preference in a particular matter.
but more interesting, the origin:
Old English sōna ‘immediately.’
😛#2018-07-2715:58alexmillerThen I’ll go with no :)#2018-07-2721:01ghadinotes on git deps with
https with authentication is not supported yet by clj (https is 65% of github's traffic)
ssh authentication via Java is a usability trashfire (some KEX methods unsupported)
The terminal is not interactive in clojure, so you cannot accept unseen public host keys - this might be ok, but surprising.
Esoteric .ssh/config options can break cloning because the underlying java ssh libraries misread the .ssh/config file
ed25519 keys supported via ssh-agent, but not via ~/.ssh/config entries
Porting to shell out to git would help all of these issues, if remains compatible with Windows support#2018-07-2721:01ghadinotes I've previously pasted here ^#2018-07-2812:36gfrederickswould support for ~ in paths in tools.deps (or parts of it? e.g. just the user file) be a welcome feature? I'd be happy to work on it#2018-07-2812:47alexmillerMy first reaction is no#2018-07-2812:47alexmillerBut feel free to file a jira and I’ll think about it#2018-07-2814:02gfredericksroger thanks#2018-07-2815:26gfrederickscan git deps specify a subdirectory of the repo?#2018-07-2815:39gfredericksI'm making a library called user.clj and all it does is contain a user.clj file that loads other user files#2018-07-2815:39gfrederickse.g., ~/.clojure/user.clj, and it could probably search the classpath for user-<n>.clj#2018-07-2815:39gfredericksand you could disable it with env variables#2018-07-2815:42mfikesGit deps can point at a subdirectory. I can’t recall the key. Maybe :deps/root or some such #2018-07-2914:38gfredericksWelp, I did it.
Nobody told me not to.
https://github.com/gfredericks/user.clj#2018-07-2915:01dominicmAre they merged? Or something else? #2018-07-2915:13gfrederickswell right now it only supports a single file, so that's not an issue#2018-07-2915:14gfredericksif I can think of a performant way of searching the classpath for other files, then (once defined what's in scope (`user.autoload.*`?)) I'd probably just load them in alphabetical order#2018-07-2915:14gfredericksI don't think merely saying "all user.clj files will be loaded!" is possible because AFAIK clojure just loads an arbitrary one, so there's no guarantee it would be mine#2018-07-2915:15gfredericksso the whole approach assumes that the rest of your user files are named something different#2018-07-2915:15gfredericks(note that ~/.clojure/user.clj would not, in general, be on the classpath)#2018-07-2915:19dominicmWhat does the library do then? I'm a bit confused! #2018-07-2915:19dominicmI thought this was a new version of ./util#2018-07-2915:20gfredericksoh no, not at all#2018-07-2915:20gfredericks> The immediate problem it's meant to solve is that the user-level deps.edn file can't reference $HOME-relative paths as a way of providing your own user.clj directly.#2018-07-2916:39dominicmI see... So this is an alternative to having an alias in your own ~/.clojure/deps.edn pointing at a dir?#2018-07-2916:57gfredericksyes, especially because doing it that way means you need an absolute path#2018-07-2916:57gfredericksso there seems to be no way to make it relative to $HOME#2018-07-3002:37cflemingWhy does t.d.a have the org.clojure/clojure dep specified as provided?#2018-07-3002:57seancorfieldThat seems correct to me: it's always going to be run in a context that already has some version of Clojure loaded. #2018-07-3002:58seancorfield(shouldn't most Clojure projects assume Clojure is "provided"?)#2018-07-3004:09richiardiandreaThe only exception IMHO is cljs self-host which would not require any dependency on clojure#2018-07-3004:45cfleming> shouldn’t most Clojure projects assume Clojure is “provided”?
Maybe, but none of them do.#2018-07-3004:48cflemingIt means that if I get the classpath for t.d.a from Maven or whatever, it won’t contain Clojure, which it clearly needs.#2018-07-3005:02seancorfieldYeah, I've tried to drum up support for a movement to change the Clojure dep to "provided" in projects but... inertia... 🙂#2018-07-3005:20cflemingThinking about it though, none of those libs work without Clojure so it really is a dependency for them.#2018-07-3005:31seancorfieldIt can be argued both ways. Clojure itself is kind of a weird "dependency" because in most projects, the choice of Clojure version comes first and then all the other dependencies are added. So, in that sense, Clojure is "provided". But, yeah, you can also argue Clojure "really is a dependency" -- but it's often going to be a different version for each of the other dependencies...?#2018-07-3005:33alexmillerI think you have to separate app and lib contexts here too#2018-07-3005:33alexmillerApps should def declare#2018-07-3005:34alexmillerGenerally you don’t want your libs deciding the Clojure version of the context they’re used in#2018-07-3005:34seancorfieldYeah, I was kinda coming at this from the app p.o.v.: it chooses the Clojure version and the libs just deal with it 🙂#2018-07-3005:35alexmillerWhich is effectively what happens#2018-07-3005:35alexmillerAs top deps have priority#2018-07-3005:36alexmillerSo in app context, their choice is most important#2018-07-3005:36seancorfieldPretty much all the libs out there should use "provided" with Clojure and apps then drive it. But that doesn't seem to get much traction 🙂#2018-07-3005:37alexmillerYeah, I don’t think that’s a battle worth fighting#2018-07-3005:58cflemingBut t.d.a is an app, right?#2018-07-3006:01seancorfieldI've only ever used it as a library...#2018-07-3006:06seancorfieldThe clj script invokes one of the -main functions in t.d.a. tho' (resolve tags, generate manifest, make classpath) but I'm not sure I'd consider tho' an app tho'?#2018-07-3006:42alexmillerIt’s both in this case I think#2018-07-3006:44alexmillerBut brew-install builds an uberjar so it’s really the app-maker#2018-07-3016:39kennyHow are you guys debugging dependency conflicts?#2018-07-3016:40ghadiclojure -Stree for a start @kenny#2018-07-3016:41kennyDoesn't show the decision made in selecting the dependency.#2018-07-3016:45ghadifrom what I've seen: the chosen dependency should only appear once in that tree, and its only parent is the driver of version selection#2018-07-3016:47kennyCorrect. But I'd like to know the other parent dependencies that also have the conflicting child dependency.#2018-07-3019:36alexmillerThere is a hidden verbose mode that sheds some additional light#2018-07-3019:38alexmillerclj -Sdeps ‘{:aliases {:v {:verbose true}}}' -A:v#2018-07-3019:39alexmillerLikely to be confusing to some degree as it’s mostly designed to let me debug stuff right now#2018-07-3019:40alexmillerI make zero promises about it’s existence or operation :)#2018-07-3019:44j0nihehe do I detect a tiny hint of revelation regret 😄#2018-07-3020:13alexmillerJust making it clear that is a not even alpha feature :)#2018-07-3022:56gfredericksI think for that we have to drop down to the hebrew alphabet and call it an aleph feature#2018-07-3023:17cflemingIf we require people to do clj -א:v I’m pretty sure not many will find it.#2018-07-3023:19gfredericksperfect#2018-07-3023:43cflemingSeveral Cursive users have requested that Cursive show alongside the available aliases whether they’re system, user or project aliases. I can’t do that at the moment because there’s no way to tell which config file is which (short of something awful like guessing based on which is under the user’s home or something).#2018-07-3023:44cflemingOne thing that would fix that would be if -Sdescribe returned:
:config-files { :system "/usr/local/Cellar/clojure/1.9.0.381/deps.edn" :user "/Users/colin/.clojure/deps.edn" }
#2018-07-3023:44cflemingAny love for that idea?#2018-07-3023:49seancorfieldChanging -Sdescribe at this point will break a bunch of other tooling that relies on the format of the (clojure-env) call I think?#2018-07-3023:49gfredericksstill alpha!#2018-07-3023:50cflemingNo doubt.#2018-07-3023:50seancorfieldAlso, there's no reason that t.d.a. is limited to those three specific EDN files -- you can merge any number of EDN files to produce your environment...#2018-07-3023:50cflemingI guess following Rich’s latest talk that should be :config-files2#2018-07-3023:50seancorfield🙂#2018-07-3023:50cflemingYou could put any number of keys in that map too.#2018-07-3023:51gfredericks-Sdescribe2#2018-07-3023:51cflemingI love it.#2018-07-3023:51ghadiit's not a breaking change at all#2018-07-3023:51ghadiyou cannot change :config-files but you could enrich the map#2018-07-3023:52seancorfield:config-file-map 🙂#2018-07-3023:53ghadiI ran into only being able to do -Sdeps once per invocation today#2018-07-3023:53cflemingHehe, or :config-file-type indexed the other way around, so the file path goes to a keyword describing its type.#2018-07-3023:53ghadithis is related to the "how many config files are being merged?" question#2018-07-3106:21dominicm@cfleming I've opened a ticket for this. It was a map at some point. #2018-07-3106:50alexmillerCurrently the config files are treated generically by the code as a list of dep sources. We now have several places that need to know the semantic meaning of different config files so they can be included or not, located, etc. (-Srepro, local dep path resolution, env, etc). This also interacts with whether the install level config is an explicit file or embedded as a resource inside tda. Something definitely needs to change but I need a clear space to sit down and think through all the details, so it’s coming.#2018-08-0216:09martinklepschAre there any restrictions to where tools.deps will look for the :sha provided?#2018-08-0216:10ghadiwhat do you mean?#2018-08-0216:11alexmillerwhere else would it look other than the git repo?#2018-08-0216:13martinklepschclj -Sdeps '{:deps {org.clojure-grimoire/lib-grimoire {:git/url "" :sha "767d68f605e5ec2ab740ae42cfc7b90a2948713c"}}}'
This fails with Manifest type not detected when finding deps for org.clojure-grimoire/lib-grimoire in coordinate {:git/url "", :sha "767d68f605e5ec2ab740ae42cfc7b90a2948713c"}
In a more involved setup it fails with the following error/stacktrace :#2018-08-0216:14martinklepsch#2018-08-0216:16ghadithe stack trace is a bit revealing... did you make edits to the file in .gitlibs/libs/... ?#2018-08-0216:16alexmiller767d68f605e5ec2ab740ae42cfc7b90a2948713c doesn’t look like a commit in https://github.com/martinklepsch/lib-grimoire ?#2018-08-0216:17ghadiI cloned and found that commit#2018-08-0216:17alexmilleroh yeah, weird - github didn’t find it#2018-08-0216:18ghadi(We should consider making the checkout dirs readonly)#2018-08-0216:18dominicm@ghadi how do you make dirs read only? 😄#2018-08-0216:18dominicmI guess you can chmod on linux, to prevent self-harm#2018-08-0216:19alexmillerI got your second error trying to run your first command#2018-08-0216:20martinklepschhaha#2018-08-0216:21martinklepschI didn’t do any manual edits in .gitlibs (as far as I’m aware at least 😛)#2018-08-0216:21ghadiI found it super easy to accidentally jump to source in Emacs to a gitlibs SHA dir.#2018-08-0216:22ghadimake edits, hit save, break Content-Addressing guarantees#2018-08-0216:23dominicmI've done the same thing to real jars#2018-08-0216:23dominicmI had to uninstall zip so vim wouldn't let me :w inside a jar.#2018-08-0216:23dominicmI should fix it properly#2018-08-0216:22alexmillerI think the second error is the real error (the MissingObjectEx)#2018-08-0216:22alexmillerand then after that, it looks like you have a checkout directory, but it didn’t finish so it’s missing files (like the manifest files)#2018-08-0216:22martinklepschI don’t even know how to jump to source so guess I’m safe 😄#2018-08-0216:22martinklepschshould I try nuking .gitlibs?#2018-08-0216:22alexmillerI think you’ll see the same problem again#2018-08-0216:23ghadino, not if Alex repro'ed immediately.#2018-08-0216:23alexmiller^^#2018-08-0216:23martinklepschright, fair enough 🙂#2018-08-0216:24bronsai can confirm reproing w/o any particular setup#2018-08-0216:24martinklepschInterestingly there are some files in that SHA’s gitlib dir (LICENSE and README)#2018-08-0216:25alexmilleryeah, it got part way done#2018-08-0216:25ghadi➜ lib-grimoire git:(develop) git cat-file -t c7cd4f50871b20a580a66e77a34bfa9d336fe430
fatal: git cat-file: could not get object info
#2018-08-0216:25ghadirepush#2018-08-0216:25alexmillerI guess a) there is either something weird in this repo or a bug in jgit#2018-08-0216:26alexmillerand b) in this kind of a failure, we should nuke the checkout dir#2018-08-0216:27alexmillerticket for b) in https://dev.clojure.org/jira/browse/TDEPS welcome#2018-08-0216:28alexmillera) is worth a bit more investigation#2018-08-0216:28alexmillermaybe there’s a force push or something weird there?#2018-08-0216:28ghadiThere is something weird in this repo.#2018-08-0216:28martinklepschWould love a tarball deps procurer btw 😛#2018-08-0216:28ghadi➜ lib-grimoire git:(develop) git cat-file -p HEAD^{tree}
100644 blob 3bc48030cba1b0f250f8b3d2cb0a2635f5195339 .gitmodules
100644 blob 7689f30efd6dcaadfb4033a444fa1c3f1848dba3 LICENSE
100644 blob c87d64a78fcaebbdfe02f188168a3270962869a1 README.md
040000 tree 4c51b4bd0c9980a782b29286cdeefdb49e7bd899 bin #### THIS
100644 blob 54f9aeba51d4b4a61c9ad3de046e770859a217c1 deps.edn
100644 blob f60a2b75223a9c8739d525ea62ba48aeeab878b0 project.clj
040000 tree 97a002c7db97cd89dc74606b77de343d2dbc878d resources
040000 tree 28ffc92968bebaf4930b08ff59d9ff5a3a5c297b src
040000 tree f4581b6f130115e5f0035945fc4ef6036076ba88 test
➜ lib-grimoire git:(develop) git cat-file -p 4c51b4bd0c9980a782b29286cdeefdb49e7bd899 ## Drill in
160000 commit c7cd4f50871b20a580a66e77a34bfa9d336fe430 hooks
100644 blob f3acf21ed723011f2586416e842ce41683bf3f29 install-hooks.sh
#2018-08-0216:29ghadiThe second command cats out the SHA for the tree representing the bin directory, and it's pointing to a commit blob mysteriously#2018-08-0216:29ghadiI expect only blobs and trees to be children of trees#2018-08-0216:30martinklepschI did force push to that branch, and it might have even been after I tried to use another revision from that repo — but seems weird that this would still replicate on a system that hasn’t had a clone yet?#2018-08-0216:32alexmillerI agree with Ghadi that that looks like a bad git structure#2018-08-0216:33ghaditry to overwrite bin/hooks and recommit it#2018-08-0216:36martinklepschDeleted bin/ entirely & commited as 9810f7b20c1a676060f939cfd8f2f785f4dea3c7 but still same issue#2018-08-0216:36ghadifetching....#2018-08-0216:36martinklepschdid you have something else in mind when you wrote “overwrite”?#2018-08-0216:37ghadirepo looks good to me, you updated to 9810f7b20c1a676060f939cfd8f2f785f4dea3c7?#2018-08-0216:37martinklepschoh no wait#2018-08-0216:38martinklepschseems to work now!#2018-08-0216:38ghadicool#2018-08-0216:38ghadiI don't know how that commit object got into the repo!#2018-08-0216:39martinklepschjust saw an exception and was assuming it’s the same but it was a different one 🙂#2018-08-0216:39ghadigit fsck didn't complain at all...#2018-08-0216:39ghadibut that might not be looking for this prob#2018-08-0216:40alexmillerI was just doing that myself :)#2018-08-0216:41martinklepschgotta say, I’m impressed by your git-fu 😄#2018-08-0216:41alexmilleryou’ve impressively horked this repo, kudos :)#2018-08-0216:42martinklepschdon’t look at me, I’m pretty sure it was @arrdem 🙂#2018-08-0216:42alexmillerfair enough :)#2018-08-0216:42alexmillerjust change my “you” ref via force-push#2018-08-0216:44martinklepschan aside to this: are there any plans to add more procurers? A tarball procurer would have been cool in this situation 🙂#2018-08-0216:45dominicmI've been hoping that a procurer extension mechanism would appear, off the back of the work to locate the owner of a namespace.#2018-08-0216:45alexmillerseems pretty far down the priority list#2018-08-0216:46alexmillerwhy not just the existing jar support?#2018-08-0216:46dominicmI imagine because github provides a tar.gz/zip on the releases page#2018-08-0216:46martinklepsch👆 this#2018-08-0216:47alexmillerwell, feel free to file an enhancement, preferably with a use case#2018-08-0216:51ghadiseems like a weird use-case#2018-08-0216:53alexmillerI can envision how to get a repo into this state via the plumbing api but I find it harder to envision how it happened via the porcelain#2018-08-0216:53alexmillerI assume some kind of bad rebase#2018-08-0216:53alexmillerbut geez#2018-08-0216:55alexmillerand ghadi, it does seem like Git handles this better than jgit so maybe you’d never even see it shelling out#2018-08-0217:35lilactownI’m trying to patch juxt/pack to allow me to pass in an alias to use while generating a jar#2018-08-0217:36lilactownthis is the relevant bit of code that is catching fire atm:
(let [deps-map (tools.deps.reader/slurp-deps (io/file deps))
resolve-args (tools.deps/combine-aliases
deps-map
(get-in parsed-opts [:options :alias]))]
(tools.deps/resolve-deps deps-map resolve-args)
#2018-08-0217:37lilactownI’m currently getting: org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact org.apache:apache:pom:1
3
#2018-08-0217:37dominicm@lilactown try adding :mvn/repos to your deps.edn#2018-08-0217:37dominicmSee pack issues, just eating dinner else I'd give the full snippet #2018-08-0217:39lilactownthat seemed to fix the error 🙂 thanks @dominicm#2018-08-0217:41dominicm@alexmiller to solve this, I'm considering inlining the system deps.edn into pack, do you think that would be a problem? #2018-08-0217:41lilactown(and it works!)#2018-08-0217:49alexmillerNot as long as you’re willing to roll with future changes#2018-08-0218:20dominicmHappy enough for now. #2018-08-0218:41seancorfieldI originally had the system deps baked into boot-tools-deps then took it out in favor of calling (clojure-env) to get the list of deps.edn files to include -- which takes care of the :mvn/repos issue.#2018-08-0218:41seancorfieldDoes pack not use the system/user deps.edn files?#2018-08-0218:42dominicm@U04V70XH6 I don't want to do that in pack, I explicitly want to ignore the user deps.edn file, and I can't do that without (clojure-env) returning a map for the deps.edn files it considers.#2018-08-0218:43seancorfieldYou could at least assume the first entry in that vector was the system deps tho'...#2018-08-0218:44dominicmBut that isn't part of the api, it could break at any point. I'd feel like a bad citizen depending on a coincidence rather than a contract.#2018-08-0218:48seancorfieldI thought we were "guaranteed" that the vector of deps files was in order? Or do you mean that whole -Sdescribe thing isn't documented/guaranteed? /cc @alexmiller#2018-08-0218:49dominicmI recall asking and being told the order wasn't guaranteed. I asked when lein-tools-deps did this.#2018-08-0218:53seancorfieldI must admit, I find that very surprising since order does matter ...#2018-08-0218:54dominicm@U04V70XH6 but being the first isn't attributed to system in anyway. A least-important override could be added at any time.#2018-08-0220:17alexmilleryou can be guaranteed that they are in order: install, user, project, -Sdeps#2018-08-0220:18alexmillerhowever, the last 3 could all independently be either missing or present#2018-08-0220:18alexmillerand in future the first may possibly not exist#2018-08-0220:31seancorfieldThank you!#2018-08-0221:06dominicmIf this guarantee is given, then I think the open ticket to convert it to a map is not needed.#2018-08-0221:50seancorfieldWell, you still couldn't tell the difference between system + user and system + project (except, perhaps, by inspection of the actual paths?) and that was something @U0567Q30W was after I believe.#2018-08-0221:51seancorfieldAnd you also wouldn't necessarily be able to tell the difference between system + user and user + project at the point where the system deps moved into the library itself (depending on how that is handled).#2018-08-0221:52alexmillerI think it’s useful to differentiate#2018-08-0221:52alexmillerfor other needs too#2018-08-0221:55seancorfieldAdding a new key to the result of -Sdescribe with some sort of map explaining where the deps files came from seems useful regardless of other concerns. I think if the system deps file moves into the codebase, then indicating that somehow in the list of config files, with a way to get the raw data structure, will also become important for tooling.#2018-08-0302:06cflemingThe latest version of Cursive also bundles the system deps.edn BTW#2018-08-0218:49dominicm@alexmiller I might be being silly, but I don't see a way in the public api to slurp a dep which is a java.net.URL, e.g. io/resource. slurp-edn depends on being a file, and canonicalize-all-syms is a private function, so I cannot utilize it.#2018-08-0220:24alexmillercould probably break slurp-edn into something smaller that takes a reader (although the file-ness is being used to construct a good error message there)#2018-08-0220:24alexmillercanonicalize-all-syms is not public as it may go away#2018-08-0220:25alexmillerthere’s now a canoncialization step built into the extension apis#2018-08-0220:25alexmilleryou can of course invoke it via the private var if you want to call it for now#2018-08-0321:10emergence@alexmiller (or anyone else in the know) - I'm pulling down a company Clojure library from a private github repo via deps.edn -- had to tweak a few things to get auth to play nice, but now I have another issue: is there a way to indicate which jar file it will use?#2018-08-0321:10emergenceI have one that will include test dependencies and am looking to use that version instead#2018-08-0321:20ghadiwhat did you tweak for Auth @emergence?#2018-08-0321:21emergenceI had to comment out my IdentityFile line in .ssh/config#2018-08-0321:23ghadiyeah this is a known issue#2018-08-0321:23ghadiwe have a potential resolution for a host of auth related issues#2018-08-0321:25emergenceI saw there is an open bug around this#2018-08-0321:21alexmillerdeps.edn does not have any way to include test scoped transitive deps#2018-08-0321:22emergenceSorry I wrote incorrectly, I meant test namespaces not transitive deps#2018-08-0321:22alexmillerI was confused by you saying jar file above#2018-08-0321:22alexmillerin tandem with github repo#2018-08-0321:23alexmilleror rather, I am still confused. perhaps you can back up. :)#2018-08-0321:24emergenceso in my deps.edn I have something like
mycompany/depname {:git/url "#2018-08-0321:24emergenceAnd forgive me if I'm overlooking something silly I'm trying tools.deps.alpha for first time s#2018-08-0321:25emergenceI want to access test namespaces from depname#2018-08-0321:26alexmillerin that project’s deps.edn, you could include :paths ["src" "test"]#2018-08-0321:26alexmillerbut there is no way to do that externally#2018-08-0321:27alexmillerso I think my answer is still, you can’t do that#2018-08-0321:29emergenceThat's still helpful -- the project is still running off a project.clj file so I can see about merging in a secondary deps.edn for my purposes.#2018-08-0321:31alexmillerit is possible to use a :deps/root key to point to a subdir holding a deps.edn that included the test, but that’s likely to confuse people#2018-08-0321:34emergenceIf I did :deps/root "test" and stuffed the deps.edn in test/ do you think that would work?#2018-08-0321:35emergenceand then specified the appropriate relative paths#2018-08-0321:41emergenceEither way I'll play around with it and see what works for my team -- thanks for spending a few mins chatting on this 🙂#2018-08-0420:07lilactownhow would I go about adding a dependency (Java) that uses gradle (via Github)?#2018-08-0420:07lilactownno pom to be found#2018-08-0420:09dominicm@lilactown generally it's deployed to a maven repository#2018-08-0420:09dominicmEven if it's using gradle#2018-08-0420:10lilactownah. in this particular instance it’s not, since the code I want is actually a part of an application#2018-08-0420:11lilactownhttps://github.com/AsamK/signal-cli#2018-08-0420:11lilactownmy hope is that I could bring it in as a dependency and play with it in clojure#2018-08-0420:15dominicmI don't think so in this case. t.d.a doesn't have a reader for gradle files.#2018-08-0420:19lilactownI guess I could fork it and generate a pom?#2018-08-0420:21dominicmor just install it locally via gradle#2018-08-0510:53michalhi. is there any way to force clj to run in offline mode (to not fetch newest snapshots in particular) ?#2018-08-0520:28alexmillerCurrently, no, but planning to add#2018-08-0515:27lilactownis add-lib going to make it’s way into master?#2018-08-0520:27alexmillerYes#2018-08-0623:19kennyI have org.slf4j/slf4j-nop in my :exclusions for a dependency but it is still included in the tree printed out via clj -Stree. org.slf4j/slf4j-nop is a second level dependency (i.e. a dependency that my dependency depends on). Why is the org.slf4j/slf4j-nop not getting excluded?#2018-08-0623:36hiredmanwhat does -Spath say?#2018-08-0623:50kennyIt also includes org.slf4j/slf4j-nop.#2018-08-0700:11kennyDeleted .cpcache and it fixed it.#2018-08-0707:24dottedmagSounds like a bug. Why it was not invalidated on its own?#2018-08-0711:50alexmillerThe only cache staleness bug I’m aware of is with local deps#2018-08-0711:51alexmillerWhere downstream deps.edn that change don’t force a cache invalidation#2018-08-0717:32martinklepschI would really like to fix https://dev.clojure.org/jira/browse/CLJ-2037 but if I understand correctly it’s tricky with the current registry structure since keywords can’t carry additional metadata. Would be happy about any suggestions. Maybe @alexmiller can also comment to what degree this feature warrants a restructuring of the registry.#2018-08-0717:34martinklepsch(I assume the patch Felix Andrews describes in his comment won’t work with specs like (s/def ::thing ::other-thing) but I haven’t fully checked IIRC)#2018-08-0717:37alexmillerdefinitely on board with fixing it, but not going to happen until Rich has a chance to get back to spec work, and that will likely be a while#2018-08-0717:41martinklepschGot it. Guess I’ll run with a fork that has a fix for this issue then.#2018-08-0718:33dnolenis it not possible to pass JVM opts via :main-opts?#2018-08-0718:35dominicmThat's what jvm-opts is for? #2018-08-0718:39dnolenah thanks, I keep looking for stuff in the guide instead of the reference#2018-08-0718:41dnolenaliases don’t compose across deps.edn files right?#2018-08-0718:47seancorfieldNo, later aliases replace earlier ones with the same name.#2018-08-0718:47seancorfield(I think that's the safer behavior, TBH)#2018-08-0718:48dnolenyeah not saying it’s desirable#2018-08-0718:49seancorfieldHeh, I've wished for composable aliases from time to time... but always stop myself from creating a JIRA ticket 🙂#2018-08-0812:20rickmoynihanis it (or will it ever be) possible to put specific paths from inside a git dep on the class/resource path, without needing a deps.edn file at the project root?#2018-08-0812:20rickmoynihanthinking for resource deps#2018-08-0812:26alexmillerNo?#2018-08-0812:37dominicm@rickmoynihan you could probably use gitlibs directly!#2018-08-0812:42rickmoynihan@dominicm: how do you mean?#2018-08-0812:43dominicmI suppose not if you wanted them on the classpath 🤔#2018-08-0812:43dominicm@rickmoynihan https://github.com/clojure/tools.gitlibs if io/file is okay for you though. Obviously I don't have the context of your use case.#2018-08-0812:45rickmoynihanThanks hadn’t seen gitlibs before#2018-08-0812:46rickmoynihanit’s ok I think :local/root is better for this anyway#2018-08-0813:01alexmillertools.deps uses gitlibs for its git deps#2018-08-0818:58richiardiandreahas anybody run into problems with OpenJDK 9 and retrieving deps?#2018-08-0818:59richiardiandreaI have a weird failure in a cljs-tooling build: https://travis-ci.org/clojure-emacs/cljs-tooling/jobs/413721389#2018-08-0819:15lilactownis there a way to tell the CLI tools to “just download dependencies”?#2018-08-0819:15lilactowntrying to tune my docker build#2018-08-0819:17dominicmAs opposed to doing what#2018-08-0820:19richiardiandreareporting the full error here, is it a known issue or it is worth a Jira?#2018-08-0820:32richiardiandreait looks very similar to https://dev.clojure.org/jira/browse/TDEPS-20#2018-08-0820:56alexmillerthat’s different#2018-08-0820:56alexmillerthe problem here is buried - “Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact mount:mount:pom:0.1.13 from/to clojars (https://repo.clojars.org/): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target”#2018-08-0820:56alexmillerthis is a certificate problem re the JDK being used and clojars#2018-08-0821:01alexmilleropenjdk9 does not install with any root certs (I think this is fixed in 10)#2018-08-0821:02alexmillerhttps://github.com/travis-ci/travis-ci/issues/9368#2018-08-0821:49richiardiandreaoh cool ok, thanks a lot Alex#2018-08-0820:24ghadi@lilactown you can put RUN clojure -Spath in your dockerfile#2018-08-0820:25ghadishould have the intended effect, if an oblique approach#2018-08-0821:39lilactown@ghadi thanks! I think that almost worked… for some reason my uberjar command still downloads and installs all it’s deps 😭#2018-08-0821:39ghadilein uberjar?#2018-08-0821:40lilactownnot quite:
:uberjar
{:extra-deps
{pack/pack.alpha
;; {:local/root "/Users/will/Code/pack.alpha"}
{:git/url ""
:sha "c444ae3af083b0bfa68160daac576607232f556c"}}
:main-opts ["-m" "mach.pack.alpha.capsule" "dist/lilactown.jar"
"-a" ":server" "-m" "lilactown.core"]}
#2018-08-0821:40ghadiAll its deps, or just the pack deps?#2018-08-0821:41lilactownI think it’s just the pack deps#2018-08-0821:41ghadiRUN clojure -A:uberjar -Spath#2018-08-0821:41ghadishould do the the trick to handle that#2018-08-0821:42lilactownhere’s my relevant dockerfile stuff:
COPY deps.edn .
RUN clojure -A:server:uberjar -Spath
COPY . /usr/app
RUN clojure -A:uberjar
#2018-08-0821:43lilactownhrm. I wonder if changing it to -R:server:uberjar -Spath would change it#2018-08-0821:43lilactownseems more correct anyway#2018-08-0821:43ghadiare you setting WORKDIR /usr/app in the dockerfile?#2018-08-0821:43ghadi-A subsumes -R#2018-08-0821:44lilactownyes, I’m setting workdir too#2018-08-0821:47ghadi(I like multistage builds)
I'm surprised L27 redownloads things, to be honest. All should get downloaded on L21#2018-08-0821:48lilactownyeah me too 😕#2018-08-0821:49ghadiofftopic for this room, but I wouldn't trust openjdk-8-alpine until OpenJDK fully supports Alpine#2018-08-0821:49ghadiJDK 11 has official support, IIRC#2018-08-0821:49lilactown🤔 I haven’t noticed any problems yet. anything in particular I should watch out for?#2018-08-0821:50ghadinot sure tbh. I've also used it in the past. Now I use jlink --add-modules to create slimmer packaged JVM#2018-08-0821:52ghadiFROM openjdk:10 as jdk
RUN jlink \
--strip-debug \
--compress=1 \
--vm=server \
--output /tmp/linkedjdk \
--add-modules ,jdk.management,jdk.unsupported,java.xml.bind,java.xml.ws.annotation
FROM debian:stretch-slim
COPY --from=jdk /tmp/linkedjdk /jdk/
# This is temporarily necessary as it appears the new OpenJDK TrustStore is incomplete
# Apparently, AWS's Step Function endpoint uses a different signing root than S3
COPY --from=jdk /docker-java-home/lib/security/cacerts /jdk/lib/security/
COPY my.jar /MYCOMPANY/
ENTRYPOINT ["/jdk/bin/java"]
#2018-08-0821:52ghadithat's my dockerfile @lilactown#2018-08-0821:53lilactownslick#2018-08-0821:53lilactownsaving this for later 😉 I remember you talking about this last week I think#2018-08-0821:55ghadidepending on what modules you need, you can get by with less#2018-08-0821:51ghadiI'd assume it would manifest as a hard VM crash. But if it works, it works!#2018-08-0917:44richiardiandreauhm, this baffles me, what am I doing wrong?
clojure -Srepro -Sdeps '{cljs-tooling {:mvn/version "0.3.0-SNAPSHOT" :exclusions [org.clojure/clojure org.clojure/clojurescript]}}' -Spath
src:/home/arichiardi/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/home/arichiardi/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/home/arichiardi/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar
#2018-08-0917:45richiardiandreain particular I am not seeing the dep I want to include#2018-08-0918:28dominicm@richiardiandrea is it the same for other dependencies, or just that one? I'm wondering about -Srepro and -Sdeps#2018-08-0918:30richiardiandreauhm dunno let me try smth else#2018-08-0918:31ghadiYeah I'm not sure how those interact together...#2018-08-0918:31richiardiandrea$ clojure -Srepro -Sdeps '{mount {:mvn/version "0.1.13"}}' -Stree
org.clojure/clojure 1.9.0
org.clojure/spec.alpha 0.1.143
org.clojure/core.specs.alpha 0.1.24
#2018-08-0918:31ghadi-Srepro might negate Sdeps#2018-08-0918:31richiardiandreaoh, maybe I need :deps#2018-08-0918:32richiardiandreariiiiight#2018-08-0918:32richiardiandrea$ clojure -Srepro -Sdeps '{:deps {mount {:mvn/version "0.1.13"}}}' -Stree
org.clojure/clojure 1.9.0
org.clojure/spec.alpha 0.1.143
org.clojure/core.specs.alpha 0.1.24
mount/mount 0.1.13
#2018-08-0918:33ghadiOh, yeah that would help#2018-08-0918:34richiardiandrealol#2018-08-1004:38steveb8nis there a lib that provides a watcher (e.g. re-runs tests or a specific fn when it sees a file change) that works with deps.edn? I can’t find anything#2018-08-1005:02seancorfield@steveb8n Wouldn't that just be any command-line watcher than can (re-)run another command when files change?#2018-08-1005:02steveb8nprobably but I’ve only used the test watcher#2018-08-1005:02steveb8nwatchers#2018-08-1005:02seancorfieldThe clj script is a one-shot runner that loads deps.edn and runs whatever :main-opts are specified.#2018-08-1005:03steveb8nand the test watchers are all built for lein/boot#2018-08-1005:03seancorfieldThe lein/`boot` test watchers are built to re-run tests -- so it's very tightly coupled.#2018-08-1005:04steveb8nI could build my own, load it into classpath from a gist but I’d rather avoid the work if it’s already been done#2018-08-1005:05seancorfieldI actually think test watchers are a bit of an anti-pattern...#2018-08-1005:06seancorfieldThey sort of imply an external process -- external to your editor and your REPL -- that you have to keep watching while you are editing.#2018-08-1005:10seancorfieldWhen I'm working on code, I'm running code in the REPL, and I can run any tests in my editor with a hot key.... I wouldn't expect to re-run my entire test suite on every minor code change and file save.#2018-08-1005:11seancorfieldAnd my workflow usually involves evaluating code without even saving files anyway ¯\(ツ)/¯#2018-08-1005:12seancorfieldI can evaluate forms to the REPL without saving the file so why would I bother with that step? 🙂#2018-08-1005:13steveb8nI agree although I’m generating a diagram using dorothy and I have to switch from the layout ns to the driving ns every time I want to see the change#2018-08-1005:13steveb8nwould be much faster during dev to see updates without the switch#2018-08-1005:13dominicmKrei is already watching the classpath for changes. No reason it couldn't run tests in a classloader or something. #2018-08-1005:13steveb8ncool. I’ll take a look#2018-08-1005:14dominicmIt doesn't do that right now though. #2018-08-1005:14steveb8ndoh, bait-in-switch 🙂#2018-08-1005:16steveb8nin that case I’ll carry on switching ns for now. maybe one of the test runners will get this soon.#2018-08-1005:16steveb8nthanks @seancorfield @dominicm for the thoughts#2018-08-1005:20seancorfield@steveb8n One option to consider is to have a (comment ...) form in your layout ns that calls functions in your driving ns, then you can do everything in one namespace.#2018-08-1005:22seancorfieldI tend to have a (comment ...) block at the end of a lot of my ns's that let me run code to setup/test/teardown the code in that ns.#2018-08-1005:27steveb8nI have that but most of my iterative changes are in the driven ns, not the driving ns. I’ll try that idea in the driven ns though - might work#2018-08-1005:29steveb8nthat works! good idea, thanks. now re-loading the driven ns invokes the fn in the driver ns#2018-08-1005:30steveb8nthe simplest ideas are always best#2018-08-1006:11seancorfield@steveb8n Cool. I've gotten a lot more productive by dropping (comment ...) blocks all over the code base so I can stay in one file and run whatever code I need (and because it isn't compiled at load time, you can have circular references in such blocks). I use it for code that starts/stops Components, runs tests, sets up state etc.#2018-08-1006:12steveb8nyeah, it was the circular ref that was blocking my brain from trying that. great trick#2018-08-1006:12steveb8nmy fingers thank you#2018-08-1010:16thheller@alexmiller I'm looking into using the tools.deps.alpha CLJ API as the primary dependency resolver for shadow-cljs (was using pomegranate before). resolve-deps works nicely but I noticed that make-classpath just concats all paths together without sorting them properly. does the CLI itself also use this behavior?#2018-08-1010:17thhellerorder shouldn't matter but in case there are any conflicts on the classpath I'd rather have something deterministic rather than relying on the hash ordering of the lib-map#2018-08-1010:17dominicm@thheller This is intentional, my understanding is that the statement is "order doesn't matter".#2018-08-1010:18thhelleryeah just might be me running into issues with pomegranate where order definitely mattered#2018-08-1010:18thhellerbut it didn't resolve version conflicts properly, so shouldn't happen in tdeps#2018-08-1012:31alexmillerOrder should only matter if you have two dependencies that contain the same thing#2018-08-1012:32alexmillerAnd if you have that something is wrong#2018-08-1012:32alexmillerMaven uses order for selection but we use version#2018-08-1013:48mhuebertis there any way to have multiple git dependencies in the same git repo, which depend on each other? eg. say I publish https://github.com/acme/MonsterLib, which contains two projects, A and B, in subdirectories, so we have /A/deps.edn and /B/deps.edn, and B depends on A. For local dev, I can put a :local/root entry in /B/deps.edn which points to "../A", but (as far as I know) this won’t be properly required by a consumer of B#2018-08-1013:49mhuebertmaybe a clearer way of asking is, “is there a way for tools.deps to resolve relative :local/root (or equivalent) dependencies in a git repo”#2018-08-1013:49dominicm@mhuebert I think you're looking for :deps/root#2018-08-1014:31rickmoynihanSo does :deps/root allow you to treat a sub project (i.e. a sub directory with a deps.edn file) within a git repo as a specific dep, instead of the top level project directory? Or am I misunderstanding?#2018-08-1014:37dominicmThat's exactly it#2018-08-1013:50mhuebertlooking for docs on that#2018-08-1013:51mhuebertnothing mentioned on https://clojure.org/reference/deps_and_cli#2018-08-1013:52alexmillerI don’t think there are any docs but it’s a thing#2018-08-1013:52mhuebertwould the entries be the same as with :local/root? ie. just put relative paths there#2018-08-1013:52alexmillerYes#2018-08-1013:53mhuebertsounds like exactly what I was hoping for#2018-08-1013:53alexmillerRelative to local/root#2018-08-1013:55dominicm@alexmiller @mhuebert I think exactly as :local/root is wrong here, it would be just "A/" alone I think.#2018-08-1014:15mhueberthmm, I must still have something wrong. I’ve made a simple example, a tree like:
.
├── App
│ └── deps.edn
└── Monster
├── A
│ └── deps.edn
└── deps.edn
(repo: https://github.com/mhuebert/deps-root-test)
App/deps.edn depends on {:local/root "../Monster"}, which depends on {:deps/root "A"}, but if I run clj -Spath in App I get Coordinate type not loaded for library monster/A in coordinate #:deps{:root "A"}#2018-08-1014:16dominicm@mhuebert you need to do a :git/url too, alongside the :deps/root#2018-08-1014:18mhueberthmm. can Monster ever depend on the current version of A?#2018-08-1014:18mhuebertif they are in the same repo#2018-08-1014:20mhuebert(if you need the sha, which you don’t have until you have made the commit that would presumably contain that link)#2018-08-1014:20dominicmhttps://dev.clojure.org/jira/browse/TDEPS-74 I believe this would solve it.#2018-08-1014:21mhuebertyes, if :local/root was relative to its own location instead of the current working directory, then I could use that#2018-08-1014:22alexmillerit can’t be relative to itself#2018-08-1014:22dominicm@alexmiller relative to the manifest, no?#2018-08-1014:23alexmillerwhen you include a local dep by location, it can be relative to the project including the dep#2018-08-1014:23alexmillerinstead of relative to cwd#2018-08-1014:24dominicmI think you're both saying the same thing, aren't you? Or am I misunderstanding?#2018-08-1014:24alexmillermaybe :)#2018-08-1014:24alexmillerI’m just trying to restate clearly for myself mainly#2018-08-1014:25mhuebertwhat I am wondering is, if that jira ticket was solved, could Monster/deps.edn contain a {:local/root "A"} pointer to the A subdirectory, and resolve for a consumer of Monster, eg. via :local/root (in the example above, App) or a git dep?#2018-08-1014:27alexmillermaybe?#2018-08-1014:32mhuebertI’m not sure how else one could consume multiple projects (which may have dependencies among themselves) from the same git repo#2018-08-1014:34mhuebertmonorepo woes.#2018-08-1014:41mhuebertI’ve updated the example to be more simple/realistic,
├── App ;; user-land, wants to use project A
│ └── deps.edn
└── Monster ;; library-land, a repo or local dir
├── A
│ └── deps.edn
└── B
└── deps.edn
the motivating case being that you want to publish project A, which depends on project B, in the same Monster repo.
The test case is, can App depend on A via git or local/root, and have tools.deps successfully follow the dependency to B.#2018-08-1014:51alexmillerwhat’s not being tracked at all in the code is the directory basis and you want that to shift as you examine different local or git dirs#2018-08-1014:52alexmillerso that it’s relative to the currently focused lib#2018-08-1014:52alexmillerand then use it to resolve relative dirs#2018-08-1015:00alexmillerwhich was actually kind of the intent of :deps/root#2018-08-1015:00alexmillerlocal deps don’t actually use deps/root right now to detect the manifest (git deps do)#2018-08-1019:59dominicmDoes it make sense for it to do so? Wouldn't you just provide a local path which included the deps/root?#2018-08-1020:17alexmilleryes, it makes sense, and no, I don’t think you would do that. You want to be able to specify a deps/root relative to the local root I think? but this is actually the less interesting part of the problem.#2018-08-1105:36dominicmI don't understand what additional functionality, beyond specifying a subdir, is provided by deps/root. What behaviour would be different?#2018-08-1015:00alexmillerso there are actually several issues here#2018-08-1218:42thheller@alexmiller does tools.deps support providing maven mirrors? in lein/pomegranate I could set :mirrors. Can't find anything related in tools.deps sources?#2018-08-1218:42alexmillerNot yet#2018-08-1218:50thhellerm2/settings.xml is read either right?#2018-08-1220:51seancorfield@thheller It reads the repos/credentials from that -- https://github.com/clojure/tools.deps.alpha/blob/884d7ae5b9c228ff795e4385291708102f1cd46d/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj#L71#2018-08-1308:36martinklepschI haven't looked around much, but just in case this is a known thing, some folks building docs for cljdoc locally get an error like this
Error building classpath. /var/folders/gm/p9fh4d3n5m11nnyjrsry1cm80000gq/T/cljdoc-expound-expound-0.7.2-SNAPSHOT6141675865145266918/impl-src is not a relative path
Does that look familiar to anyone? Are :paths required to be relative?#2018-08-1323:35richiardiandreasorry maybe an FAQ but can tools.deps read deps from a local folder?#2018-08-1323:36richiardiandreaoh :local/root awesome#2018-08-1405:45isaaccan clj handle java files that in paths?#2018-08-1405:47dominicmNo#2018-08-1405:47dominicmOr rather, not in source files. If they're from Maven it's okay. #2018-08-1405:48isaaccan i call javac from clj?#2018-08-1406:21dominicm@isaac absolutely.#2018-08-1406:21dominicmI do this in pack.#2018-08-1703:12alex-dixonGoing off isaac’s question from earlier, is there a good workflow for mixed java and clojure projects using deps.edn?#2018-08-1710:33jetzajaccan I add a lib that consists of multiple jars on disk?#2018-08-1712:07alexmillerGenerally, no. Typically a lib consists of a single artifact#2018-08-1712:08alexmillerThere are ways to hack it by listing the jars explicitly in :paths though#2018-08-1718:10lilactowndo multiple -Sdeps merge?#2018-08-1718:30lilactownanswer: nope#2018-08-1806:52cfleming@alexmiller Ugh, is it expected that users will be able to add jars via :paths? I’ll have to add support for that if so.#2018-08-1808:47alexmillerI would not consider that to be normal practice#2018-08-1915:30sveri@alexmiller Hi, how about native support for windows, did you come to work on it?#2018-08-1916:04alexmillerI’ve done a port to power shell but have not had time to finish all the build and release stuff, which interacts with some of the script#2018-08-1916:05alexmillerHoping to be able to refocus on that soon and finish it off#2018-08-1918:44sveriOk, thanks for your work. I was trying to use cli on WSL, which works, but fighweel main for instance needs an x11 instance which I don't have installed right now.
#2018-08-2008:01levitanonghi all, is there a way to run clojure such that it only downloads the dependencies and exits? Similar to lein deps. Use-case is in docker, where caching is done by diffing USEd files.#2018-08-2009:02delaguardoclojure -e "(clojure-version)"#2018-08-2009:02delaguardothis will download everything, print out your clojure version and exit with exit code 0#2018-08-2009:07levitanong@delaguardo haha wouldn’t clojure -e "(System/exit 0)" work equally well in that case?#2018-08-2009:08delaguardosure)#2018-08-2009:08delaguardoI’m just copying my configuration where I’m using printed value as a part of deploy report)#2018-08-2010:37levitanongOkay, thanks @delaguardo!#2018-08-2010:54dottedmagclojure -e '' works too#2018-08-2012:52alexmillerclojure -Spath will work#2018-08-2016:26levitanong@dottedmag oh that’s nice and concise!
@alexmiller would clojure -Sdescribe also work?#2018-08-2016:26levitanongI ask because that would end up printing a lot less than -Spath#2018-08-2016:26ghadiclojure -Spath >/dev/null#2018-08-2016:26ghadisufficient for a dockerfile ^#2018-08-2016:28levitanongHaha @ghadi that’s an interesting solution, but I think I’d prefer clojure -e '' to that.#2018-08-2016:28ghadithat will launch the JVM twice, which is why alex suggested -Spath#2018-08-2016:36levitanongoooh, so like, once for getting the deps, and then another for launching the repl?#2018-08-2016:36ghadisi#2018-08-2016:36ghadinot a big deal#2018-08-2016:42levitanongstill good to know how things work! Thanks for explaining 🙂#2018-08-2117:35ghadihttps://clojurians.slack.com/archives/C03S1KBA2/p1534872342000100#2018-08-2117:35ghadihttps://clojurians.slack.com/archives/C03S1KBA2/p1534872369000100#2018-08-2118:39ghadi(Of course the absence of a convenience doesn't hold me back at all when I have a choice)#2018-08-2118:55alexmillerif you’re not using aliases that have M or O, then why do you care?#2018-08-2119:02ghadia test alias is a good example of when you want to launch a repl with deps but without (let's say) main options for some test-runner#2018-08-2119:09ghaditldr: because I am using an alias with M / O#2018-08-2123:19alexmillerI’m going to file this in the tiny violin category#2018-08-2123:52ghadiI'll take a tiny little 🎹 @alexmiller #2018-08-2123:57alexmillerGranted#2018-08-2200:54cflemingNow I have to create a Tiny Violin tag in my Github tracker.#2018-08-2214:04dominicm@alexmiller is this something you would want to support?
{:git/url "../"
:deps/root "sub-dir"
:sha "abcabc"}
Essentially mono-repos with version pinning.#2018-08-2214:04dominicmI'm getting:
Error building classpath. Destination path ".." already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path ".." already exists and is not an empty directory
at org.eclipse.jgit.api.CloneCommand.verifyDirectories(CloneCommand.java:257)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:189)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:49)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
at clojure.tools.gitlibs.impl$ensure_git_dir$fn__850.invoke(impl.clj:109)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
at clojure.tools.deps.alpha.extensions.git$eval894$fn__896.invoke(git.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:238)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:168)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:152)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:215)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
which I suspect is due to how the folder name is being chosen.#2018-08-2214:07dominicmoh, weird, I can't supply an absolute path either, it is forced to be a relative one.#2018-08-2214:08dominicmLooks like paths aren't supported at all: Error building classpath. org.eclipse.jgit.transport.TransportLocal cannot be cast to org.eclipse.jgit.transport.SshTransport#2018-08-2214:08dominicmI'll open JIRA tickets#2018-08-2214:21alexmillerlocal git repos were intentionally out of scope#2018-08-2214:22alexmillercould possibly be added, but I would rank that below a lot of other things to work on#2018-08-2214:22alexmillerso I’ve classified these as minor enhancements in jira#2018-08-2214:29dominicmsounds very reasonable to me. I just don't touch the priority fields, based on what I think I'm supposed to do.#2018-08-2216:07onetom@dominicm the whole point of a monorepo is that you don't have to worry about the versions, because you are supposed to ensure that on every commit all modules/libraries/apps still build and work in your monorepo 😕#2018-08-2216:07dominicm@onetom I know 😞 sometimes we put things into life support though, quite useful under those conditions.#2018-08-2216:10onetomI was wondering how can I control the refresh period of -SNAPSHOT dependencies.
Or is it possible to pin them to a specific version?
For example, I have all these versions:
$ ls -lah ~/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/*.jar
-rw-r--r-- 1 onetom staff 135K Aug 14 20:07 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-20180813.135516-2.jar
-rw-r--r-- 1 onetom staff 137K Aug 15 19:16 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-20180814.194035-3.jar
-rw-r--r-- 1 onetom staff 140K Aug 21 08:29 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-20180818.154912-4.jar
-rw-r--r-- 1 onetom staff 140K Aug 21 08:29 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-SNAPSHOT.jar
I would be happy to just stick to figwheel-main-0.1.8-20180818.154912-4.jar until im on a slow internet connection and extra network traffic can cause several seconds of extra delay#2018-08-2216:12dominicmI think that "0.1.8-20180818.154912-4" works as a version#2018-08-2216:13onetomI was thinking that maybe one solution would be to just use a git dependency in this case, but then it also depends on SNAPSHOT versions of figwheel-core and figwheel-repl
https://github.com/bhauman/figwheel-main/blob/master/deps.edn#L6-L7#2018-08-2216:14onetom@dominicm thanks, indeed it does.
so i can just specify explicit versions of the transitive dependencies too and it should be fine.#2018-08-2216:15onetomstill, it would be good to be able to disable automatic SNAPSHOT updating somehow. iirc i could do that in boot#2018-08-2216:19dominicmin boot I pass --offline#2018-08-2216:21onetomyeah, that's the option i was thinking about!#2018-08-2618:21bhaumanIs there a central list of tools.deps tools like packr?#2018-08-2618:34dominicmJ/w Do you mean pack.alpha? #2018-08-2618:22bhaumanI remember there was one called mayven or something like that and I can’t seem to find it out there on webs#2018-08-2618:22seancorfieldThere's a partial list on repo's wiki#2018-08-2618:24bhaumanDo you remember the mayven project? I though metosin did it but nope.#2018-08-2618:24bhaumanIt’s not on the wiki#2018-08-2618:24bhaumanthanks for pointing out that page though#2018-08-2618:25seancorfieldhttps://github.com/clojure/tools.deps.alpha/wiki/Tools (for other folks wondering about the URL)#2018-08-2618:26seancorfieldI don't recall mayven (but I'm terrible with names)#2018-08-2618:26bhaumangood pin#2018-08-2618:26bhaumanand its a horrible name to search for#2018-08-2618:27seancorfieldYeah, Bing isn't turning up anything for me with mayven in it.#2018-08-2618:45alexmillerI recall it but don’t have a link. I don’t remember it being a tools.deps thing?#2018-08-2619:11bhaumanI thought it was a tools deps thing but that may be why I’m not finding it#2018-08-2619:51dominicmhttps://github.com/danielsz/meyvn#2018-08-2619:53dominicm@bhauman ^#2018-08-2621:37bhauman@dominicm Thanks!#2018-08-2701:17steveb8nhas anyone else used cognitect.test-runner in CI? I’m finding it returns status zero when there are test failures#2018-08-2701:26seancorfield@steveb8n Are you sure? I use it with clojure.java.jdbc and I'm pretty sure I get a non-zero status when there are failures.#2018-08-2701:26seancorfieldAre you using clojure or clj?#2018-08-2701:26steveb8nI’m using clojure#2018-08-2701:26steveb8nis that a possible explanation?#2018-08-2701:27steveb8nI can reproduce on my laptop also. have looked at source and it doesn’t jive#2018-08-2701:27seancorfieldclojure is the underlying script so it should be safe(r) anyway.#2018-08-2701:27seancorfieldAre you on an older version of clj/`clojure`?#2018-08-2701:28steveb8nnope. 1.9.0 in deps.edn#2018-08-2701:29steveb8nI might try debug in Cursive and see if that explains it#2018-08-2701:29steveb8nweird#2018-08-2701:31cfleming@steveb8n I think @seancorfield is talking about the clj/`clojure` CLI scripts, not Clojure itself.#2018-08-2701:31steveb8noh good point. I’ll check#2018-08-2701:32steveb8n1.9.0.375#2018-08-2701:35steveb8nwhen I run from repl I see correct results so I think Sean is right in that this is maybe the scripts masking the system/exit value#2018-08-2701:41steveb8njust updated clojure using brew. same result sadly. will keep digging#2018-08-2701:43steveb8njust spotted this commit. I’m on an old version of the test runner. testing new sha now#2018-08-2701:44steveb8nthat was it. thanks for the sounding board 🙂#2018-08-2701:47steveb8ninteresting challenge to keep the SHA’s in readmes current when a jar/snapshot not available in clojars. not sure how that can be improved.#2018-08-2702:25alexmillerrub some script on it#2018-08-2702:30seancorfield@steveb8n Sorry, stepped away. Glad you got it figured out. Not sure why a SHA is any harder to keep up to date than a Maven/Clojars dependency? You still have to go look at the project and see what's changed...#2018-08-2702:33steveb8nfair point. I agree. I guess the exception is snapshots but I generally don’t use them anyway. also the clojars build/badge provides the latest without a readme update I believe. that’s one way that clojars beats shas for readme maintenance. anyway I’m good and I’ll check latest shas from now on. thanks all#2018-08-2704:34levitanongWhat’s the difference between putting some code in "-e '(foo)'" and putting the same code in some namespace with -main, and putting it in an alias main-opt?#2018-08-2705:57steveb8n@alexmiller interesting turn of phrase. I might “borrow” that#2018-08-2818:35richiardiandrea@seancorfield question about clj-new's generators - how do I handle the case where I want to modify a file in the project root, like a deps.edn?#2018-08-2818:35richiardiandrea(and is it common/good practice?)#2018-08-2818:45seancorfield@richiardiandrea That sounds like a nice enhancement -- to expose the prefix argument somehow. Feel free to open an issue.#2018-08-2818:45richiardiandreaok#2018-08-2818:45richiardiandrea@seancorfield oh I think this is similar to https://github.com/seancorfield/clj-new/issues/8#2018-08-2819:29johnLooks like it's been implemented here: https://github.com/juxt/pack.alpha/blob/master/src/mach/pack/alpha/inject.clj#2018-08-2818:47seancorfieldThe main obstacle right now is that you can't specify the prefix (or any other parameters to generate-code under the hood), mostly because the command line processing isn't very sophisticated. Hence the open issue about looking at tools.cli for some of this.#2018-08-2819:34richiardiandrea@seancorfield sorry other question, can I build a template that accepts a single path in clj-new ?#2018-08-2819:40seancorfieldI'm not quite sure what you're asking there?#2018-08-2819:48seancorfieldIf your template function accepts [name & args] instead of the default [name], then any additional arguments you provides on the command-line are passed in...#2018-08-2819:49richiardiandreait seems like I always receive an error directly from clj-new if I pass the project name as my-template-test for instance#2018-08-2819:50seancorfield$ clj -Sdeps '{:deps {clj-new {:git/url "" :sha "108f27159501f8ce12564b398b0ea50def3892b1"}}}' -m clj-new.create template myt/thing
...
$ vi thing/src/clj/new/thing.clj # add & args and print them out
$ clj -Sdeps '{:deps {clj-new {:git/url "" :sha "108f27159501f8ce12564b398b0ea50def3892b1"} my/thing {:local/root "./thing"}}}' -m clj-new.create thing another.generated.thing with args
Generating fresh 'clj new' thing project.
args= (with args)
$ #2018-08-2819:50seancorfieldclj-new does not accept a single-segment name#2018-08-2819:50richiardiandreaoh ok that's what I feared#2018-08-2819:50seancorfieldIt must either be a multi.segment.name or a qualified/name#2018-08-2819:51seancorfieldPhil always regretted allowing that bad practice for lein new 🙂#2018-08-2819:51richiardiandreaoh ok got it#2018-08-2819:51seancorfieldYou can always say my-template-test.core to get the Leiningen behavior.#2018-08-2819:51richiardiandreatoo bad because it feels natural for JS devs (which are my audience now)#2018-08-2819:52richiardiandreayeah#2018-08-2819:52richiardiandreahave to explain that 😄#2018-08-2819:53seancorfieldThe docs say:
> The project name should be a qualified Clojure symbol, where the first part is typically your GitHub account name or your organization's domain reversed, e.g., com.acme, and the second part is the "local" name for your project (and is used as the name of the folder in which the project is created). An alternative is to use a multi-segment project name, such as foo.bar (the folder created will be called foo.bar and will contain src/foo/bar.clj).#2018-08-2819:53seancorfieldBTW, note above how I generated a template and then used it via :local/root 🙂 Gotta love the whole clj / deps.edn thing!#2018-08-2819:55richiardiandreayeah I am using the :local/root thing, it is a nice thing#2018-08-2819:56richiardiandreathe thing is, JS users do not care/are used to the Java wait of packaging - therefore they consider it an additional burden..#2018-08-2819:56richiardiandreasome people here at least 😄#2018-08-2819:56richiardiandreaso the naming now for them feels Java-y#2018-08-2819:56seancorfieldJS people should learn the benefits of globally unique names 🙂#2018-08-2819:57seancorfieldI have zero sympathy! :rolling_on_the_floor_laughing:#2018-08-2819:58richiardiandreawell, they have a group as well but very few people use it#2018-08-2820:01seancorfieldI will admit that when generating a new template, the qualified/multi-segment thing doesn't work as expected. If you use a qualified/name then the qualified part is ignored. If you use a multi.segment.name you get an illegal project (src/clj/new/multi.segment.name.clj). That's a bug that I haven't opened an issue for yet because there are restrictions on how template functions are named, including their namespace, so I'm not sure how best to resolve that conflict.#2018-08-2820:02seancorfieldThe simplest solution would be to allow simple names for templates but... ¯\(ツ)/¯#2018-08-2820:10richiardiandreaalso I don't see any util to get only the myproj from something like my.long.group/myproj#2018-08-2820:14seancorfieldIf you generate an actual project for my.long.group/myproj the qualifier part goes into the namespaces: (! 978)-> clj -Sdeps '{:deps {clj-new {:git/url "" :sha "108f27159501f8ce12564b398b0ea50def3892b1"}}}' -m clj-new.create app my.long.group/myproj
Generating a project called myproj based on the 'app' template.
Tue Aug 28 13:13:34
(sean)-(jobs:0)-(~/clojure)
(! 979)-> tree myproj
myproj
|____.gitignore
|____.hgignore
|____CHANGELOG.md
|____deps.edn
|____doc
| |____intro.md
|____LICENSE
|____README.md
|____resources
| |____.keep
|____src
| |____my
| | |____long
| | | |____group
| | | | |____myproj.clj
|____test
| |____my
| | |____long
| | | |____group
| | | | |____myproj_test.clj
#2018-08-2821:52richiardiandreauhm this does not seem to work in my template...I always get: Could not create directory /home/arichiardi/tmp/my.long.group/myproj. Maybe it already exists? Use -f / --force to overwrite it..
Super weird I am doing the same thing that your app is doing#2018-08-2822:50richiardiandreaoh found the problem, :name is actually used by ->files! didn't expect that 😄#2018-08-2820:14seancorfieldThat's how Leiningen and Boot work.#2018-08-2820:14richiardiandreayes that works but I would like to have {:project-name ...} in my data#2018-08-2820:14richiardiandreaworking on a patch#2018-08-2820:15seancorfield{:project-name ...} in what data?#2018-08-2820:16richiardiandreathe template data#2018-08-2820:16richiardiandreatogether with {:year ... :name ...}#2018-08-2820:16seancorfieldAh, yes, the template generator could do with a lot of enhancements.#2018-08-2820:16richiardiandreathis is something I really need, so patch will be coming#2018-08-2820:17seancorfieldThe app generator has :name for (project-name name)#2018-08-2820:17seancorfieldYou might consider sending a similar patch to boot-new since it's pretty much the same code, as I recall?#2018-08-2820:18seancorfieldThe template already has :raw-name by the way -- which is the whole name.#2018-08-2820:18seancorfieldhttps://github.com/seancorfield/clj-new/blob/master/src/clj/new/template.clj#L9-L14#2018-08-2820:19seancorfield(but the template template does not have all that -- and it probably should https://github.com/seancorfield/clj-new/blob/master/src/clj/new/template/temp.clj )#2018-08-2820:20richiardiandreakk will see what I can do#2018-08-2820:20richiardiandreamaybe I can just roll my own template#2018-08-2823:32cflemingAm I correct in thinking that there’s no way right now to get the CLI to output the actual Java command that would be executed?#2018-08-2823:32cflemingIs that cached, or is it only the classpath that’s cached?#2018-08-2823:58cflemingNever mind, poking through the code, I see the .libs, .cp, .jvm and .main files.#2018-08-2902:56alexmillerYou’re correct in that there is no method to dump the actual full Java command#2018-08-2902:56alexmillerBut it’s assembled from those pieces#2018-08-3001:34cflemingI don’t really understand the rationale for having separate alias types, e.g. R, C, O, M#2018-08-3001:35cflemingWhy not just have a single alias type and merge in whatever it contains? What’s the use case for separating them?#2018-08-3002:06kenny@cfleming I find it useful to have a :test alias that includes test dependencies and :main-opts, which I can optionally enable to actually run the tests.#2018-08-3002:16alexmilleryou may have heard that Rich likes taking things apart#2018-08-3002:51cfleming@kenny But if you had a single alias for that that would work anyway.#2018-08-3015:40kennyAh yes, you're right. That simplifies things. I have no use case for the others now.#2018-08-3002:51cflemingThe dependencies would get used when calculation the classpath, and :main-opts would get picked up when running.#2018-08-3002:53cflemingYou don’t need the concept of separation for that.#2018-08-3002:53cfleming> you may have heard that Rich likes taking things apart
Sure, but I hope he also has reasons for that. I haven’t seen any for this particular case, which is why I asked.#2018-08-3002:54alexmillerthe idea is that they can be used for different parts of the lifecycle and you can choose which part you’re affecting. Or you could say R,C,O,M are the simple part and A is the easy part#2018-08-3002:55cflemingBut surely the different parts of the lifecycle will just get picked up as needed anyway?#2018-08-3002:56cflemingIn Kenny’s test example, the :main-opts will just get ignored when building a classpath.#2018-08-3002:56cflemingIt seems like you’d only use that when you want to remove specific fields when applying an alias, and I can’t imagine a case where that would be useful.#2018-08-3002:56alexmillerbuilding a classpath involves both the R and C parts#2018-08-3002:57alexmillerthis is not a part of the design that I feel strongly about and I’m not going to give you an impassioned defense for it#2018-08-3002:58alexmilleras you know, sometimes decisions that seem important early seem less so later#2018-08-3002:58cflemingOk, thanks.#2018-08-3002:59alexmillerfwiw, I only use A these days#2018-08-3003:00cflemingYeah, that’s what I would do too. I’m mostly wondering how much support I should be adding for this to Cursive, or if I should just assume everything is implicitly A#2018-08-3003:00cflemingI think I’ll do that until someone complains about it.#2018-08-3003:00alexmillerI think that would get you far#2018-09-0408:42pesterhazyIs there a better way to get a new snapshot build than rm -rf .cpcache ~/.m2/repository/com/bhauman?#2018-09-0409:19delaguardoclj -Sforce should ignore .cpcache But why you want to delete your local maven repo?#2018-09-0409:20delaguardojust curious#2018-09-0409:29dominicm@pesterhazy @delaguardo I'm not certain, but I believe there is delegation to maven for this. So the usual 24h period is in place, regardless of ignoring the cpcache.#2018-09-0409:49pesterhazy@delaguardo when a new version gets uploaded to maven, I need to download it. By deleting the local cache I make sure the files get re-downloaded#2018-09-0409:50pesterhazyJust deleting .cpcache won't be sufficient AFAIK#2018-09-0409:54dominicm@pesterhazy j/w, do boot/lein/mvn provide options for this?#2018-09-0409:55delaguardohm… you can not upload to maven new release version (without -SNAPSHOT suffix) so when maven coordinate is changed in deps.edn (e.g. 1.2.0 -> 1.2.1) it will download it. For snapshoted versions it works differently: -SNAPSHOT suffix is replaced with the timestamp and after uploading new snapshot in maven you should have two with different timestamps.#2018-09-0409:55delaguardoall those cases already handled by tools.deps and there is no need to drop your local cache#2018-09-0409:56pesterhazyI was clearly talking about snapshot builds in my original question, no?#2018-09-0409:57pesterhazylet me check the timestamp theory#2018-09-0409:58delaguardojust try with -Sforce without deleting .m2#2018-09-0409:59pesterhazyI will, thanks!#2018-09-0409:59pesterhazyPretty sure that boot supports boot -u to re-download new snapshots#2018-09-0410:00pesterhazyWanted to check, but the web page doesn't list the command line arguments - it refers you to boot -h - and installing boot takes 5 min#2018-09-0410:00delaguardoboot -u is for upgrading boot tool itself#2018-09-0410:00pesterhazyOk, it's boot -U#2018-09-0410:00pesterhazyoh no#2018-09-0410:00delaguardosame, but for the latest snapshot#2018-09-0410:01pesterhazylooks like I misremembered#2018-09-0410:49alexmillertools.deps uses daily update strategy for SNAPSHOTs currently so won’t update new ones even with -Sforce#2018-09-0410:50alexmillerI’m planning to change that but haven’t gotten to it yet#2018-09-0410:52alexmillerSo for the moment you would need to both delete this portion of your .m2 and use -Sforce (no need to drop ALL of your .m2 though)#2018-09-0410:54alexmillerOr you could gen a pom with -Spom and use mvn -U dependency:tree to have Maven do it (followed by -Sforce to tell deps to update)#2018-09-0414:36pesterhazythanks for the explanation!#2018-09-0414:37pesterhazyI guess rm -rf is easy enough, it's just hard to explain to others in the team#2018-09-0415:45alexmillerYeah#2018-09-0508:53maleghastHey there, everyone... I am a little lost, but probably just being stupid; I want to create an uberjar from my tools-deps based application. Any clues, links, etc..? (My Google-Fu has failed me completely)#2018-09-0508:59dominicmhttps://github.com/juxt/pack.alpha and https://github.com/healthfinch/depstar/#2018-09-0509:07maleghastThanks @dominicm#2018-09-0509:11maleghast@dominicm - Which Jar approach would you recommend via Pack for Docker-based deployment..?#2018-09-0509:12dominicm@maleghast I recommend capsule as a good default 🙂#2018-09-0509:12maleghast🙂#2018-09-0509:12maleghastThx#2018-09-0509:49maleghast@dominicm - I may be doing something stupid (very likely), but on running the following:
clj -A:pack mach.pack.alpha.capsule foundation.jar -e build-dir --application-id foundation --application-version "$(git describe)" -m foundation.main
I can find the file -> mach.pack.alpha.capsule <- but not my jarfile... 😞#2018-09-0510:53dominicmWhat output are you getting? None?#2018-09-0510:56maleghaster, hold on...#2018-09-0510:58maleghast@dominicm I get / got this ->
https://gist.github.com/maleghast/b671e5299d709c79813ca80b4a81f7bd#2018-09-0511:29dominicmThere's a file named after the namespace, odd! #2018-09-0511:30dominicmWhat's in it? #2018-09-0511:30dominicmAlso, how does your pack alias look? #2018-09-0511:31maleghasthuh?#2018-09-0511:32dominicmThere's a file named mach.pack.alpha.capsule in that folder #2018-09-0511:32maleghastYeah, I know#2018-09-0511:33dominicmWhat is it?#2018-09-0511:33maleghastI copied your command from the Pack Github page and just changed the name of the output jarfile and the app main namespace (it's up there ^^)#2018-09-0511:34maleghastI have no idea what it is, I thought it was supposed to be there.#2018-09-0511:34dominicmNope#2018-09-0511:34dominicmWhat does your pack section in deps.Edn look like? #2018-09-0511:36maleghastEr, one sec#2018-09-0511:36maleghast:pack {:extra-deps
{pack/pack.alpha
{:git/url ""
:sha "1d455f6c591f440560c57a29e6d930004d735dcf"}}
:main-opts ["-m" "mach.pack.alpha.capsule"
"-m" "foundation.main"]}
#2018-09-0512:35mike_ananev@maleghast I recommend you to stay with lein for some time. pack.alpha, depstar and other have some lacks... one of them copy src files into uberjar, another doesn't copy resource folder into uberjar. I've investigated all of them and IMO lein is still preferable way to build uberjar.#2018-09-0513:29dominicmpack does put src files in, because that's what you generally want. If you want to AOT, that is supported with pack, but you do it as an earlier step.
I'm happy to point you at how you can do this, and we're working on improving the documentation to make it much clearer on how to do this kind of thing.#2018-09-0609:17Andreas Liljeqvistplease point 🙂#2018-09-0609:20dominicmhttps://github.com/juxt/edge/blob/master/bin/uberjar#2018-09-0609:20dominicmhttps://github.com/juxt/edge/blob/master/bin/uberjar#L17-L26 in particular#2018-09-0512:37mike_ananev@maleghast most promising tool is https://github.com/luchiniatwork/cambada#2018-09-0513:00maleghastThanks @mike1452 I will take a look... The app I am trying to UberJar is built on top of a library that @dominicm works on, that uses tools.clojure and deps.edn and as such uses neither leiningen or boot, and I am loath to add either into the mix if I don't have to. That being said, if I have to...#2018-09-0513:21maleghast@dominicm - I think that I know what's wrong with the command I pasted above...#2018-09-0513:21maleghastthis works:
clj -A:pack foundation.jar --application-id foundation --application-version "$(git describe)" -m foundation.main
#2018-09-0513:27maleghast@dominicm - yeah, that does work... However I need to figure out how to package up the CSS etc. as the app is functioning, but looks AWFUL 😉#2018-09-0513:27dominicm@maleghast yeah, I figured that the alias you had already had the mach.pack.alpha.capsule in the alias 🙂#2018-09-0513:27maleghastYeah, that's what I worked out as well...#2018-09-0513:28dominicm@maleghast the -e build_dir option is a flag to provide an extra directory to be combined into the jar. So you can build your css/js into a directory, and have it run.#2018-09-0513:28dominicmIf you're using edge, there's an uberjar script in master which has these steps in 🙂#2018-09-0513:28maleghastAh... Thanks! 🙂#2018-09-0513:28maleghast(I am using edge)#2018-09-0514:56pesterhazyMade a script for grepping in your classpath: https://gist.github.com/pesterhazy/74f6bc235dfe5f64690250d2ee84969d#2018-09-0514:56pesterhazy#2018-09-0514:58pesterhazye.g. clojure -Spath | unpack-cp, then cd to that directory and run grep from there#2018-09-0518:17mike_ananev@dominicm Did you use pack/pack.alpha? can't understand how to avoid copying source files into uberjar using this tool#2018-09-0518:39dominicmI wrote it. Is having source files in there bad? #2018-09-0710:08mike_ananevWell it depends. I think it should as parameter, like lein does ( :omit-source true) . When I put my uberjar in external untrusted environment I don't want that my source files be availabale for other people.#2018-09-0710:09dominicmI haven't looked into that kind of use case particularly, but I understand now. Skipping the :paths seems like a possible flag that can be added though, it shouldn't be too difficult.#2018-09-0719:03mike_ananevYeah. It would be great.#2018-09-0621:37gashow do I clear the cache trying to do clj -A:new figwheel-main hello-world.app --reagent but getting yesterdays cached version of fighweel.main template?#2018-09-0622:08bhauman@gas 0.1.9 is out so you can just update to that#2018-09-0622:27gasyeah I see that however also saw you updated template so tryed to run clj -Anew but keep getting old template settings?#2018-09-0623:29seancorfield@gas does clj -Sforce work to clear the cache (that clj keeps)?#2018-09-0623:31seancorfield(or you can just delete the .cpcache folder where you are running clj and that should force it to recreate any classpaths)#2018-09-0623:31seancorfieldI think Alex mentioned a bug around snapshot handling...#2018-09-0623:31gastrying -Sforce @mo where is cpcache udner home?#2018-09-0623:31seancorfieldAh, here we go https://clojurians.slack.com/archives/C6QH853H8/p1536058198000100#2018-09-0623:32seancorfieldThe .cpcache folder exists wherever you run clj#2018-09-0623:32seancorfield(it creates that in every folder that you run it in)#2018-09-0623:33gasok -Sforce didnt work.#2018-09-0623:33seancorfieldPer Alex's comment?#2018-09-0623:33seancorfield(snapshots only get checked for once a day)#2018-09-0623:34gasyeah but not sure how bhaumann figwheel-main template works as it doesnt seem to have a version. presume its goes off git?#2018-09-0623:36gasdont seem to have .cpcache where I am running clj? on a mac not in home wither. hmm found it in .clojure/.cpcache#2018-09-0623:37gasdelete ~/.clojure/.cpcache ??#2018-09-0623:37bhaumantemplates are a bit different#2018-09-0623:38bhaumanI’m releasing a new template very soon#2018-09-0623:39seancorfieldclj-new requests RELEASE or SNAPSHOT versions as part of the dependency lookup as I recall... Let me go look at the code. It's been a while since I wrote it.#2018-09-0623:39bhaumanoh that would be cool#2018-09-0623:40gas@bhauman ok - I have copied your template and works great - FYI only issue in template at moment you need react as deps for reagent config for it to work with regent 0.8.1#2018-09-0623:42seancorfieldOK, so here's what happens in clj-new...#2018-09-0623:42seancorfield...it looks for <template>/clj-template then <template>/boot-template then <template>/lein-template, in that order.#2018-09-0623:43bhauman@gas weird, I tested every config, I’m surprised#2018-09-0623:44bhauman@seancorfield so no snapshot versions?#2018-09-0623:44seancorfieldFor the clj-template, if the template name looks like a git URL or a filesystem local path, it will only use that.#2018-09-0623:44bhaumanoh well thats good#2018-09-0623:46seancorfieldOtherwise it will either request a RELEASE version or a SNAPSHOT version -- but I'm checking whether the flag to expose snapshot is in place...#2018-09-0623:46bhauman@gas it shouldn’t need react at at all#2018-09-0623:46seancorfield...right now, that flag isn't exposed in clj-new (only in boot-new ... and lein new)#2018-09-0623:47bhaumanvery good to know#2018-09-0623:48seancorfieldBTW, a "git template" looks like <url to repo>@<SHA> and the last part of the URL is considered to be the template name.#2018-09-0623:48seancorfieldSo you can pull specific versions of templates down from GitHub using that.#2018-09-0623:48gas@bhauman ok I commented out react and it works this time! Only diff is i had the #framework part in my core.cljs but dont see any react stuff in that code#2018-09-0623:49seancorfieldAh, it won't work with https://github.com/bhauman/figwheel-main-template because it ends it -template rather than just the template name (`figwheel-main`).#2018-09-0623:50bhauman@seancorfield the snapshot part won’t work?#2018-09-0623:51seancorfieldThe git dependency won't work.#2018-09-0623:52seancorfieldclj -A:new hello-world.app#2018-09-0623:52seancorfieldThat would work if that was indeed the name of the repo -- but you have -template on it.#2018-09-0623:53bhaumangotcha#2018-09-0623:53bhaumanthere’s already a figwheel-main repo 🙂#2018-09-0623:53seancorfieldI could provide an option to override the actual template name (like I do with local paths: /path/to/some/code::my-template#2018-09-0623:53bhaumanI think the git url covers this case pretty well#2018-09-0623:54seancorfieldWell, the problem is that the template name doesn't match the repo path.#2018-09-0623:54bhaumanoh#2018-09-0623:54bhaumanok#2018-09-0623:54bhaumanhmmmmm#2018-09-0623:55seancorfieldIf I added that option, you could do clj -A:new hello-world.app
#2018-09-0623:55seancorfieldI should make it consistent on both git and local deps really...#2018-09-0623:56seancorfieldAnd figure out a way to enable snapshot lookup...#2018-09-0623:57seancorfieldhttps://github.com/seancorfield/clj-new/issues/11#2018-09-0623:59seancorfieldhttps://github.com/seancorfield/clj-new/issues/12#2018-09-0623:59seancorfieldI'll work on those tomorrow afternoon or over the weekend.#2018-09-0707:43levitanongHi guys, does anyone have a cljs tools-deps library I can look at? Trying my hand at it, but somehow the library can’t see itself. Lol#2018-09-0711:36dominicmI'm interested in solving TDEPS-12, @alexmiller have you thought about how you'd like it to be done? #2018-09-0715:48isaachow to use clj cli compile java code#2018-09-0715:48isaacI have some java code in classpath#2018-09-0716:13alexmiller@dominicm I don’t think anything on that ticket is very good thinking about this problem#2018-09-0716:14alexmillerthe problem is that the assumption going into the design is that there is a 1-1 correspondence between library name and artifact#2018-09-0716:14alexmillerthis is a case where that is not true#2018-09-0716:15alexmillereverything else flows from that and without addressing that fundamental issue, there is no way to fix the issue#2018-09-0716:16alexmillerI’ve been thinking about this for a long time and I think you have to solve the “name” part or you’re not fixing anything#2018-09-0716:19alexmillerthe library name here is used for dual purposes - identification in tools.deps, and lookup in Maven (the groupId:artifactId). In many places we key maps based on #1 so that really needs a new dimension of context that is more than just #2.#2018-09-0716:19alexmillerthat means encoding the classifier into the name (so you get uniqueness of identity) and have the Maven extension know how to decode that for lookup#2018-09-0716:20alexmillerthat could be done with a name encoding, or by using something other than a symbol as the library name#2018-09-0716:22dominicmInteresting. When discussing with @malcolmsparks I dismissed the notion of using a vector or some encoding for the key.
Something not mentioned is that there's more than just classifier as a dimension in Maven, there's also type. I suspect encoding is a bad solution because it's not information dense enough. #2018-09-0716:24alexmiller{:groupId "group.id", :artifactId "artifactId", :classifier "classifier"}#2018-09-0716:37alexmillerdoing one does not preclude doing the other later#2018-09-0716:39dominicmI don't have one to hand, that's certain. In that case I will defer to your data. My objection was weak and I was erring on the side of exposing everything in the case of not having access to any data.
#2018-09-0716:40alexmillertools.deps has been out for about a year and I haven’t seen a request or question about anything other than classifiers#2018-09-0716:40dominicmIf we did that, would it be a consistent return value? I'm guessing it would involve changing the current lib map spec? #2018-09-0716:40alexmillerwhat is “it” in above?#2018-09-0716:40dominicmThe map version of the lib name.#2018-09-0716:41alexmilleryes, would change (grow) the spec and a couple things in the Maven extension canonicalization#2018-09-0716:41alexmillerI really need to talk to Rich to make a decision on that#2018-09-0716:43dominicmAt the moment I rely on it being a symbol in pack, in order to perform naming. That seems like a breaking change for me. Am I relying on a property I shouldn't? #2018-09-0716:43alexmillerthe code changes are pretty trivial. thinking clearly about the problem here is the important part.#2018-09-0716:52dominicm> choose whatever name you want with lowercase letters and no strange symbols.
I was just looking into which character was appropriate. Maven is surprisingly coy about what is allowed... #2018-09-0716:54dominicmI guess I will just have to test. #2018-09-0717:28lilactownlol so I blew out ~/.m2 and now when I try and run clj:Error: Could not find or load main class clojure.main
#2018-09-0717:44dominicm@lilactown I'm guessing because the classpath is cached#2018-09-0717:48alexmilleryes, if you manually muck with .m2, you’ll need to use clj -Sforce#2018-09-0717:48alexmillercurrently clj does not do any checks to verify that all the dirs/files on the classpath exist#2018-09-0717:49alexmillerI’m a bit on the fence about it - that could be added, but has the downside of a) adding startup time and b) not being tolerant of (for example), a missing src path on the cp. although I guess you could maybe only check the files.#2018-09-0718:30zentropeMaybe just a warning to “if in doubt, clj -Sforce” on “missing” errors?#2018-09-0718:32lilactownyeah, I didn’t know about -Sforce. that would be a helpful enhancement#2018-09-0718:38zentropeOr, could do the path checking AFTER the error occurs, in which chase “start up time” isn’t a factor.#2018-09-0718:40zentropeVerify that there are missing paths for a good error message, not re-do the whole op.#2018-09-0719:56alexmillerthe benefit of doing it before is that you can potentially fix it, not error#2018-09-0720:00alexmillerif stuff is stale, then force#2018-09-0720:01alexmillerthe question I’d like to answer is - how long does it take to do the check for a reasonable size classpath#2018-09-0723:35zentropeRight. The “after” idea is to keep the speed benefits, but provide a clearer option for the user to try: same as it is now, just slightly clearer.#2018-09-0807:24malcolmsparks@alexmiller @dominicm One example where type is important is where a project might depend on a .war file, containing some functionality. See https://codesjava.com/eclipse-maven-java-web-project for example. This is more likely to happen inside a large enterprise using J2EE, I doubt you'd find an example in 'public' repos but there are vast private ones out there. This still is about the classpath (Maven itself is primarily about classpaths, not file distribution). There are other types too: https://docs.oracle.com/cd/E19830-01/819-4712/ablgz/index.html. I don't think it's worth adding support for them in tools.alpha, but at the same time it would be good if any design around supporting classifiers can be extended (via growing) to support types as well.#2018-09-0807:24malcolmsparkshttps://clojurians.slack.com/archives/C6QH853H8/p1536336844000100#2018-09-0807:30malcolmsparksI think the 1-1 correspondence is actually quite a nice design. If you think of the artifact as the logical entity defined in the pom.xml file (with its version and dependencies), rather than an actual file in the Maven repo. 99% of cases there is a single file for the artifact, but sometimes there are also ancillary files (one per classifier), sometimes called secondary artifacts. While Maven does support depending on a primary artifact file of, say, version 1.0, and a secondary artifact of a different version, say, version 1.1, this is such a questionable practice it is unlikely to be worth supporting.#2018-09-0807:33malcolmsparksSo in conclusion, I don't think there needs to be an extended lib name of {:groupId "group.id", :artifactId "artifactId", :classifier "classifier"}. The libname, with the :mvn/version, already uniquely addresses the logical artefact (defined in its pom). There should be a way of specifying the required classifiers, perhaps with a :mvn/classifiers entry.#2018-09-0807:37malcolmsparksOne minor point is that Maven makes you a declare a dependency on each artifact file you want to download (and you just normally repeat the GAV coord with a different classifier). Therefore there might need a way of saying "this classifier please, but include the primary artefact too", and also 'this classifier please, but not the primary artefact". I think the latter case could be supported later if required via growing the design.#2018-09-0808:08malcolmsparksHaving read https://github.com/technomancy/leiningen/issues/898 now I do understand the point about tools.deps being only concerned about the classpath and perhaps extraction of native libs needs to be out-of-scope.#2018-09-0809:04dominicmSome libraries do the work themselves, http://usb4java.org/nativelibs.html#2018-09-0812:39alexmillerI can give you at least one more example that has nothing to do with native stuff#2018-09-0812:41alexmillerClojureScript relies on a few contrib libs (like tools.reader). All of hose libs get built in both a source form (and deployed with no classifier) and in aot form (deployed with :aot classifier). #2018-09-0814:53alexmillerthe idea of pushing this into the coord is interesting. The current extension SPI already supports extensions returning multiple artifacts per lib. I do wonder about whether this has impacts for walking the transitive deps? Something to look at.#2018-09-0814:56alexmillerI guess there’s still only one pom defining deps for a lib, even in the case where there are multiple classifiers#2018-09-0816:06alexmilleras I looked at it more, I like this solution and I have implemented it and committed the change for the next release - see TDEPS-12 for more#2018-09-0817:54gfredericksI was thinking again about self-contained clojure files
I realized you could have a pretty clean impl at the cost of a mildly longer shebang line, if you publish a library (ideally with a very short name) that can read deps out of lines 2->N of a file, and execute everything after that using -Sdeps
so the shebang would be roughly #! clojure -Sdeps '{:deps {lib7 {:mvn/version,"1"}}}' -m lib7.main
I think the tricky part is that lib7 couldn't do a unix exec to replace the original jvm; so maybe the cleanest thing for it to do would be to have tools.deps as a dep, and call it internally, and make its own classloader with the resulting classpath...
so
A) are there any libs yet that accomplish anything comparable to this?
B) is this crazy or impossible?#2018-09-0817:59gfredericksStartup time would be interesting#2018-09-0818:29dominicmThere's a library to make isolated class loaders from a deps edn. #2018-09-0818:29dominicmStartup time would be boot-like#2018-09-0818:31gfredericksdo you recall the library's name?#2018-09-0818:38dottedmag@gfredericks Shebangs do not get parsed into arguments.#2018-09-0818:39dottedmagAnd yes, someone has done something like that.#2018-09-0818:40dottedmagIIRC via file that can be read either as a shell script (that runs clj) or as Clojure code (where shell header gets ignored via e.g. ;). With #!/bin/sh shebang.#2018-09-0818:41gfredericksoh if you can't in any syntax pass two args to clojure then I guess that would kill that idea#2018-09-0818:41dottedmag#!/bin/sh
; exec clj .... $0
<clojure code follows>
#2018-09-0818:42dottedmagSomething like this.#2018-09-0818:42gfrederickshaving lots of deps crammed into one line is unfortunate#2018-09-0818:43gfredericksI bet some fancy bash could support multiple lines#2018-09-0818:43dottedmagMultiple lines of what?#2018-09-0818:43gfredericksdeps#2018-09-0818:44dottedmagWhy? I don't remember a requirement that shell script has to fit on a punched card 🙂#2018-09-0818:44gfredericksjust annoying when viewing and editing#2018-09-0818:44gfredericksnot a deal breaker#2018-09-0818:44dottedmagI'd generate this script instead of editing it manually.#2018-09-0818:44gfredericksgenerate it from what?#2018-09-0818:45gfredericksI guess I mean the thing I'm going for is casual self-contained shell scripts#2018-09-0818:45gfredericksnot tied to anything else#2018-09-0818:45gfredericksso generating it kind of defeats the purpose#2018-09-0818:45dottedmagand yes, should be trivial: #!/bin/sh
; CLJ="clj ..."
; CLJ="$CLJ ..."
; CLJ="$CLJ ..."
; exec $CLJ
<clojure code follows>
#2018-09-0818:46gfredericksI wonder if you could take advantage of the deps map being a noop to have it outside the comments#2018-09-0818:46gfredericksso you get syntax highlighting and editor help#2018-09-0818:48gfredericksI get Syntax error: ";" unexpected for that sort of thing#2018-09-0818:55gfredericksthis is the holy grail I think
#!/bin/sh
... magic bash
{:deps {...
...
...
...}}
... more magic bash, with exec
(ns wahoo ...)
#2018-09-0819:46borkdudeThis also uses a similar trick:
https://github.com/borkdude/balcony/blob/master/balcony.clj#2018-09-0819:46borkdudeDoes anyone have an example of how to use clj/tools-deps with AOT + uberjar?#2018-09-0819:47gfredericks":"; seems designed to avoid the error I mentioned above#2018-09-0819:48borkdudeyes, it was the shortest bash command I could find that was also valid clojure#2018-09-0819:48borkdudeI think @mfikes came up with it#2018-09-0819:51gfrederickswould "" not work?#2018-09-0819:51gfredericksI guess not#2018-09-0819:51gfrederickswhat does : mean in bash?#2018-09-1212:42souenzzoit's not plain true
it's a (def : (constantly true))#2018-09-0819:51borkdudeit’s an alias for true#2018-09-0819:52gfrederickso_O#2018-09-0819:52gfrederickssure why not#2018-09-0819:53gfredericksso your AOT/uberjar question is about asking how to take a src dir and AOT compile it and put that in an uberjar?#2018-09-0819:54borkdudeyes, with tools.deps. With lein I can find docs on that, but with tools.deps not so much I guess#2018-09-0820:10alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-09-0820:10alexmillerThere are some things there to do it#2018-09-0820:11borkdudeThanks!#2018-09-0819:54gfredericksI thought I saw an uberjaring lib at some point#2018-09-0819:55gfredericksw.r.t. AOT, it might be true that by calling compile enough you will get class files written to the classes dir#2018-09-0819:55gfredericksbut there might be lots of edge cases involved that I don't know about#2018-09-0819:56gfrederickse.g., if you have a top-level namespace that requires everything else, try clojure -e '(compile (quote that.ns))'#2018-09-0820:01borkdudeif [ ! : ]; then
echo "True!"
else
echo "False!"
fi
# prints "False!"
#2018-09-0820:07gfrederickswho wouldn't want to be able to write code like that!#2018-09-0820:10borkdudehowever, this doesn’t seem to work:
if [ : && : ]; then
echo "True!"
else
echo "False!"
fi
#2018-09-0905:05dominicm@borkdude https://github.com/juxt/edge/blob/master/bin/uberjar this does aot w/ pack#2018-09-0911:52gfredericksI figured it out!
#!/usr/bin/env bash
":"; DEPS=\
'{:deps
{org.apache.commons/commons-compress {:mvn/version "1.17"}}}'
":"; exec clojure -Sdeps "$DEPS" "$0"
#2018-09-1013:02rickmoynihanNeat. I’ve had a need for something like this this morning.
Have been meaning to ask why clojure doesn’t barf on the #!/usr/bin/env bash?#2018-09-1013:04rickmoynihan(read-string "#!/usr/bin/env bash") blows up… but clojure seems to somehow let it through…#2018-09-1013:38gfredericks#! is indistinguishable from ; AFAIK
your read-string call fails with ; as well, because there's nothing for it to return -- add a second line with a valid form and both will work#2018-09-1013:39rickmoynihanYeah scroll down #tools-deps 🙂#2018-09-1013:39gfredericksoh sorry#2018-09-1013:39rickmoynihannot at all; thanks for answering 🙂#2018-09-0911:52gfredericks☝️ that seems sufficient to let you keep your deps in the clojure-syntax part of the file#2018-09-0912:12gfredericksalso an advantage of not putting clojure in the shebang is that you don't have to give the fully qualified path to it#2018-09-0912:15gfrederickspro-tip: if you ever have to use a lib that contains a ' in its version string (which would threaten the bash quotation), use \u0027 instead#2018-09-0912:18dominicmYou can use a quote for a version? #2018-09-0912:20gfredericksprobably
{:mvn/version "12.43.01'-sure"}
I mean I haven't tried it#2018-09-0912:20dottedmag{haskell/haskell {:mvn/version "'"}#2018-09-0912:21gfredericksI put a fuller example here https://gist.github.com/gfredericks/900183dae706aae6361110b2383c8639#2018-09-0914:30borkdude@dominicm cool!#2018-09-0915:28pesterhazyWeird and wonderful #2018-09-1012:11aitemHi to all.
How i can omit source and obfuscate my target .jar file using deps.edn and clojure cli?
Previously i used leiningen and proguard and all works fine.
But in current project we are use clojure cli and juxt/pack.alpha for building .jar.#2018-09-1013:23rickmoynihan(read-string "#!/usr/bin/env bash") blows up… but clojure seems to be ok with it… This is the only bit I don’t understand, presumably the clj/`clojure` tools strip this out?#2018-09-1013:29dominicm@rickmoynihan they do not, this is in clojure.core#2018-09-1013:29rickmoynihanreally#2018-09-1013:29rickmoynihanwow#2018-09-1013:29dominicm! is technically a tagged literal which comments out the whole line#2018-09-1013:30rickmoynihanthat was my first thought… but why does read-string blow up?#2018-09-1013:30rickmoynihanahhh hold on maybe it’s because there’s no more input…#2018-09-1013:30rickmoynihanyeah that’s it#2018-09-1013:30rickmoynihanthanks#2018-09-1013:33rickmoynihan(read-string "#!/usr/bin/env bash\n 10") works (read-string "#!/usr/bin/env bash") fails#2018-09-1013:33dominicmboring answer 😉 Why is it always a boring answer#2018-09-1013:33dominicm(Because physics isn't broken)#2018-09-1013:40rickmoynihanWell Rich had remarkable foresight when he added this feature ten years ago 🙂
https://github.com/clojure/clojure/blame/master/src/jvm/clojure/lang/LispReader.java#L116#2018-09-1013:40rickmoynihanThough I guess even then this was useful, even then… more so now we have the clj tools#2018-09-1014:09dominicmThis was clearly always the plan 😛#2018-09-1015:21alexmillerRich plays the longest game#2018-09-1015:36dominicmI'm waiting for an IDE for clojure to appear in the clojure codebase.#2018-09-1016:20alexmillerNo comment#2018-09-1023:24Joe LaneOh man this comment is going to keep me up tonight.#2018-09-1103:05alexmillerMission accomplished#2018-09-1015:42sparkofreasonI need to use gen-class it implement an abstract Java class, but am having no luck getting it to load in the REPL for cursive. I assume it needs to be AOT'ed to be present on the classpath for clj. How is this accomplished with deps.edn? Or do I need a separate tool?#2018-09-1016:02sparkofreasonIf I use compile and add classes to :extra-paths I get this exception on import.#2018-09-1016:31nicola@aitem i’m curious too how distribute clojure closed source jars. How datomic is distributed? May be @alexmiller help us.#2018-09-1016:56alexmillerDon’t know - I’m not on the Datomic team :)#2018-09-1017:57nicolaWho can we ask about this?#2018-09-1018:26alexmillerask in #datomic#2018-09-1016:33seancorfield@rickmoynihan Looking at that code makes me wonder what #< is for since, right now, both in EDN and Clojure, the reader for that simply throws an exception.#2018-09-1016:43rickmoynihan@seancorfield: yes that’s a curious one#2018-09-1016:43seancorfieldI'd forgotten about ## as well -- and #! was new to me too.#2018-09-1016:43dominicm #<foo> might be something?#2018-09-1016:44rickmoynihanyeah ## was new in 1.9#2018-09-1016:45seancorfieldCan you override how #< is read? Or do you mean "reserved for some future, unknown purpose"?#2018-09-1016:46rickmoynihanI didn’t think the reader was extensible/overridable; at least not beyond tagged literals.#2018-09-1017:08dominicm@seancorfield I'm guessing a bit. #2018-09-1017:21mfikes#< is for values that have no way to be read#2018-09-1019:24andy.fingerhutWow, I had never noticed that #! special case in the Clojure reader before. I won't say I never saw it, because I am sure I looked at the linked group of source lines at least a dozen times over the years.#2018-09-1020:42borkdudewhat’s an example with #<?#2018-09-1020:48andy.fingerhutAs in, is there some object that will print as a sequence of character beginning with #< ? I don't know if there is one today.#2018-09-1020:52andy.fingerhutScanning through the Clojure implementation for print and pprint I didn't notice any such case.#2018-09-1021:10alexmillerNot to my knowledge #2018-09-1021:10alexmillerI think that’s just reserving space in the reader#2018-09-1022:49gfredericksit was used for everything prior to #object wasn't it?#2018-09-1022:58alexmillerYou think it’s that old object syntax?#2018-09-1022:58alexmillerMaybe so#2018-09-1023:11gfredericksmy interpretation was always that #<> was the way unreadable things were printed, and the #< entry in the reader was to produce the "unreadable form" exception whenever you accidentally tried to read those things#2018-09-1023:14gfrederickshere's the reader error messages from 1.2.1:
user=> (-> (Object.) pr-str read-string)
java.lang.RuntimeException: java.lang.Exception: Unreadable form (NO_SOURCE_FILE:0)
user=> (read-string "#,comma")
java.lang.RuntimeException: java.lang.Exception: No dispatch macro for: , (NO_SOURCE_FILE:0)
so I thought of the #< codepath as the way to avoid the less helpful No dispatch macro for: < message you'd get otherwise#2018-09-1023:15gfredericksand it's still useful today for reading data produced by older clojures#2018-09-1100:04johnI could have sworn I've seen the #<some-object> notation before#2018-09-1100:29gfredericks$ clojure -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.2.1"}}}' -e '(Object.)'
#<Object #2018-09-1100:39seancorfieldChanged in 1.7 Clojure 1.2.1
#<Object
#2018-09-1100:39seancorfield(the new CLI stuff makes this so easy to test!)#2018-09-1100:42gfredericksfor clojure in 1.2.1 1.3.0 1.4.0 ...#2018-09-1103:46richiardiandreaWe have a monorepo with this exact setup: https://clojure.org/guides/deps_and_cli#_using_local_libraries and sometimes we get the dreaded Manifest type not detected error...it is a local dep with a deps.edn and I wonder how I can debug what is going wrong...#2018-09-1103:50alexmillerprobably related to relative paths if I had to guess#2018-09-1103:50alexmilleris it a transitive local dep?#2018-09-1103:52alexmillerthere is a known issue that transitive local deps resolve paths in terms of your current directory, not in terms of the dep, which can lead to issues with finding the correct “dep root” (where the deps.edn would be expected)#2018-09-1103:54richiardiandrea@alexmiller yes it is a transitive dep that of one of the sub projects and one colleague has noticed that if he specifies it top level the problem goes away.#2018-09-1103:55richiardiandreaThanks then good to know the problem is known I will check Jira#2018-09-1103:57richiardiandrea@cfleming thanks this is the reason of the Cursive message ^#2018-09-1103:58alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-74#2018-09-1103:59alexmillerthat’s the placeholder for it, there might be another one that’s same thing#2018-09-1111:54cfleming@richiardiandrea Thanks for letting me know.#2018-09-1117:08dominicm@alexmiller someone in #leiningen just discovered that openjfx uses <type>pom</type> which wouldn't be supported by deps.edn#2018-09-1117:08dominicmjust as a data point#2018-09-1117:08alexmillertrue#2018-09-1117:10alexmillertools.deps is hardcoded to look only for jar extension#2018-09-1117:10alexmillerthe original example in TDEPS-12 is all jars#2018-09-1117:33dominicmYep. This was something @malcolmsparks pointed out a little later.
Pom is something that is logically consistent with classpath building. I don't know how type interacts with classifier, I suspect it's multiplicative.#2018-09-1117:35malcolmsparksNot quite. Each classifier/type combination is specified in a separate dependency element in the POM. #2018-09-1117:38malcolmsparks@alexmiller thanks for comment in TDEPS-12, I've been able to build a local version of tools.deps and patch it into my /usr/bin/clojure script, no problem. Still testing.#2018-09-1117:43dominicmIt might be that a classifier key is wrong, and it should be named otherwise, and be maps with classifier and type in. #2018-09-1117:44alexmillerusually I just add the source directory of tools.deps.alpha to the front of the lib path in my clojure script :)#2018-09-1117:45alexmillerpoms are used for determining transitive deps but are not themselves artifacts that get put on the classpath (these are independent parts of the tools.deps extension spi)#2018-09-1117:46alexmillerall classifiers in a group/artifact/version share the same pom#2018-09-1117:46alexmillerwhich is in some ways deeply weird#2018-09-1117:46alexmillerand really reinforces that GAV should primarily be about “one” artifact#2018-09-1117:47alexmillerI think you could argue pretty readily that the case in TDEPS-12 for example should be different artifactIds, not classifiers, but I don’t run the world :)#2018-09-1118:26dominicmI was thinking the same. But I did realize that having structure provides the opportunity for tooling to inspect it. Eg automatically fetching src & javadoc classifiers.
In the TDEPS-12 case, tooling has less of a place, but could feasibly be done by a maven plugin#2018-09-1118:29alexmillersource and javadoc are secondary info (just like Clojure meta) and make total sense to me#2018-09-1118:30alexmillereven things like aot as a secondary classifier make sense to me as a variant of the same artifact#2018-09-1118:31alexmillerany case where you’re putting more than one artifact from the same GAV on your classpath at the same time seems … weird#2018-09-1118:37dominicmI don't know GAV. Google isn't too revealing either, what is it? #2018-09-1118:38alexmillersorry#2018-09-1118:38alexmillerI’ve merged with the borg apparently#2018-09-1118:38cflemingGroup Artifact Version (standard Maven ID)#2018-09-1118:38alexmillergroupId/artifactId/version#2018-09-1118:38alexmillernice slack, out of order message delivery#2018-09-1118:39dominicmMy guess was surprisingly accurate. The version part didn't quite fit for me. #2018-09-1118:40dominicmJust so I'm clear, do you think javadoc should be a separate GAV with a pointer from the root? Or that context does make sense for a classifier?#2018-09-1118:57alexmillerno, I think the existing usage of classifiers for javadoc and source make sense#2018-09-1118:57alexmillerI think cases like the one in TDEPS-12 are weird#2018-09-1118:57alexmillerall those classifiers should be separate artifacts#2018-09-1120:33dominicmI wonder how hard it would be to have a tool which synchronized your Maven deps to add a profile with javadoc and source. I'd love that for stack traces.
A bit of a shame that it would be duplicative to isolate to an alias with the current classifier data shape. Not terrible I suppose though. #2018-09-1120:37dominicmI suppose you could use such a tool through -Sdeps too, but then cider would have to become aware of it... Unless it was a dependency of cider, then everyone would have it and be happy forever! #2018-09-1120:48seancorfieldNot everyone uses CIDER tho'...#2018-09-1120:49seancorfield(heresy, I know!)#2018-09-1120:53dominicmI think intellij is the only editor that doesn't. And atom. #2018-09-1121:01dominicmIntellij probably already does this, so is less interesting. Atom would miss out, but could integrate directly with the library. #2018-09-1122:06alexmillerintellij does this already#2018-09-1122:07alexmillerthat is, download and use source and javadoc for libs#2018-09-1204:12cflemingRight, Cursive does that for lein libs, it doesn’t work for deps yet#2018-09-1204:43alexmillerah#2018-09-1204:45cflemingClojure actually makes this pretty tricky, so in Cursive it works on demand - if you try to go to the source of a Java class IntelliJ will decompile it and prompt you to download the source at that point, it doesn’t do it up front since it’s basically impossible to know which artifacts have sources (most don’t in Clojure)#2018-09-1216:09isaacA javac tool & AOT tool https://github.com/gfZeng/tools.deps.task.compile#2018-09-1316:28abpWhen automating tests and builds with clojure we need to download maven dependencies to the project folder ideally, so we can use gitlab CI caching on the maven dependencies in the project folder. Is there any way to do this with the clojure shellscript and tools.deps? With leiningen we used local-repo https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L145#2018-09-1316:38alexmillerYep, use a root attribute in deps.edn of :mvn/local-repo#2018-09-1316:44abpWe just found it too, but thanks anyway 🙂#2018-09-1318:25richiardiandreaHello folks! I am a bit confused by this:
clj -C:gen:new:tmpl -Stree <- no deps is shown
clj -R:gen:new:tmpl -Stree <- deps are shown
but the doc says that -C is for building the classpath...#2018-09-1318:26richiardiandreaif I am building the classpath, why is -C not showing my deps in -Stree?#2018-09-1318:27richiardiandrea(and of course I don't have them in the classpath as well)#2018-09-1318:32hiredmanis there a reason you aren't using -A#2018-09-1318:34hiredmanmy admittedly not very deep understanding is, -A does a lot of stuff, an the other flags like -C and -R do subparts of what -A, I suspect -C only merges the paths entries not the dependency entries. using -A seems to provide most of the kind of default what you would expect from "merging"#2018-09-1318:35hiredmanthere used to be a comment (maybe there still is) the default deps.edn file that said '''#2018-09-1318:35hiredman;; make-classpath aliases (-C) affect the classpath generation, options:
;; :extra-paths - vector of additional paths to add to the classpath
;; :classpath-overrides - map of lib to path that overrides the result of resolving deps
#2018-09-1318:36richiardiandrea@hiredman my alias contains :main-opts and I do not want to use it for new and gen for now#2018-09-1318:36richiardiandreaI am launching a REPL against them#2018-09-1318:37seancorfieldBeat me to it... Yeah, @richiardiandrea -C only pulls in paths and the library overrides. -R is for dependencies (`:deps`, :extra-deps, and :override-deps I think).#2018-09-1318:37richiardiandreaok I think I missed that part of the doc, thanks a lot, I thought classpath meant both path and deps#2018-09-1318:37richiardiandreaas customary in all the previous tools 😉#2018-09-1318:38seancorfieldYou mostly want -R or -M but if you are bringing in test code you probably want -C for that 🙂#2018-09-1318:38seancorfieldWhat I recommend is not having :main-opts in the same alias as anything else -- use a separate alias that is just for running that main.#2018-09-1318:39seancorfieldFor example, in several projects I have :test to set everything up for testing and :runner to actually run the tests.#2018-09-1318:40richiardiandreaso I am using new and telling my team to just do clj -A:new:tmpl cljs-template org-scope/my-project#2018-09-1318:40seancorfieldI tend to keep all my aliases small and orthogonal -- so they can be composed 🙂#2018-09-1318:40richiardiandreabut now I am just in need of a REPL with the deps#2018-09-1318:40richiardiandreaso I do not want -M I guess 😉#2018-09-1318:40seancorfieldWhich is exactly why you want the main opts separate from the deps 🙂#2018-09-1318:41richiardiandreaAnyways, this works clojure -R:gen:new:tmpl -A:nrepl#2018-09-1415:22niclasnilssonUsing deps.edn, when developing an app with some libs locally, what’s the intended way when App -> LibA and LibA -> LibB. Is there a way to do this without tainting App’s deps.edn with App’s dependencies dependencies (where all are local)?#2018-09-1415:23dominicm@niclasnilsson :local/root I think? #2018-09-1415:23mpenetyou can have a profile with local dependencies#2018-09-1415:23mpenetI mean an alias#2018-09-1415:23niclasnilsson@dominicm Only seems to work in one “layer”, not picking things up recursively#2018-09-1415:24niclasnilsson@mpenet, yep, that’s what I’d like to avoid. It’s just boring.#2018-09-1415:24mpenetwell you can add exclusions and explicit deps to other local deps#2018-09-1415:24mpenetbut yeah, it requires some work#2018-09-1415:24dominicm@niclasnilsson I see, you mean when all 3 are local. Transitive aliases don't work, I don't think there's anyway to configure libraries to load in transitive local mode. #2018-09-1415:25niclasnilssonAnd (I think) even when LibA is local and LibB is on github for instance, App doesn’t seem to pick it up. (Unless I did something wrong)#2018-09-1416:02richiardiandreaThe local dep issues are quickly becoming a problem for our Cursive users - we spend a lot of time clicking on the error dialogs popping up - we tried adding :manifest :deps but the the use from the command line broke for some reason.
I might invest some time digging into it...#2018-09-1416:03cfleming@richiardiandrea I think you want :deps/manifest :deps#2018-09-1416:04richiardiandreaAh yep I think with put that sorry 😃 will come up with a repro first then we'll see#2018-09-1417:15alexmiller@niclasnilsson don’t forget that you can put aliases in ~/.clojure/deps.edn too and then combine them with stuff in your project. so if you had a bunch of local dep overrides, you could declare them there (without touching your project deps.edn)#2018-09-1423:13jcfSomething that just occurred to me. Languages like Ruby and Perl that have spent a lot more time optimising their CLIs have relatively succinct (but also somewhat cryptic) APIs for interactic with text. Consider the following examples, first in Ruby and the an approximation using clj:
$ echo 4 | ruby -pne '$_.to_i + 1'
4
$ echo 4 | clj -e '(inc (Long/parseLong (first (line-seq (java.io.BufferedReader. *in*)))))'
5
I'd argue that $_ is hard to remember and Google, but if you want to apply an operation to each line in some input the Ruby syntax is quite convenient (assuming you can remember it).
Are there plans on adding similar switches to the new clj executable to support process *in* with a bit less ceremony?#2018-09-1423:13alexmillerthere is one#2018-09-1423:13alexmillerjust append - at the end#2018-09-1423:15jcfHi @alexmiller! How would I change echo 4 | clj -e '(inc (Long/parseLong (first (line-seq (java.io.BufferedReader. *in*)))))' to make use of -? Sorry for being dumb but it's not immediately obvious to me.#2018-09-1423:15jcfI have used - with other programs before, but not with clj.#2018-09-1423:17jcfmain-opts
-m, --main ns-name
Call the -main function from namespace w/args
-r, --repl
Run a repl
path Run a script from a file or resource located at path
- Run a script from standard input
#2018-09-1423:18jcfI've found - in the man page for clj.#2018-09-1423:22jcfFor context I'm thinking I want to make it easier doing something like curl | jq '.users' | clj -e '...' so I can combine Clojure with other tools I have on the command-line.#2018-09-1423:22jcfI can of course do everything from Clojure, but just wondering if there's thought going into the API for passing in data to a new Clojure process.#2018-09-1500:09seancorfield@jcf Is this still too much code? (! 550)-> echo 4 | clj -e '(-> (read-line) (Long/parseLong) inc)'
5
#2018-09-1500:13seancorfieldIf you want to read every line: (! 552)-> (echo 4; echo 13) | clj -e '(while (some-> (read-line) (Long/parseLong) (inc) (doto (println))))'
5
14
#2018-09-1500:25jcfDefinitely more terse than my snippet. I like the use of read-line and a threading macro for sure!
I think we're missing a trick still by not having the switches provided by Ruby.
The whole loop-and-read boilerplate becomes redundant when you pass -n, and with -p you don't have to print…
#2018-09-1500:31seancorfieldClojure is a fundamentally different execution model tho': you're building a classpath, firing up a JVM, compiling a set of forms, and then running the compiled code.#2018-09-1500:31seancorfieldWith Ruby, it can just take the expression and interpret it repeatedly...#2018-09-1500:33seancorfield...for Clojure, you'd need some in-JVM process that takes your expression and does the looping/reading for you -- you can't do it at the script level.#2018-09-1501:10seancorfieldOK, @jcf how about this for Ruby-like conciseness? (! 584)-> (echo 4; echo 13) | clj -Apne -e '($ (inc (Long/parseLong $_)))'
#'user/$
5
14
#2018-09-1501:12seancorfield$ is a macro defined by the pne alias that takes the following expression and evaluates and prints it repeatedly in the context of $_ bound to subsequent (read-line) calls...#2018-09-1501:13seancorfieldAs for the pne alias... be afraid... be very afraid... (! 585)-> cat deps.edn
{:aliases,{:pne,{:main-opts,["-e","(defmacro,$,[&,body],(list,'while,(list,'when-let,'[$_,(read-line)],(let,[v,(gensym)],(list,'let,(vector,v,(list*,'do,body)),(list,'println,v))),'$_)))"]}}}
#2018-09-1501:14seancorfield(you can't use backtick easily with clj due to bash issues so you have to laboriously build the entire body with list and vector -- and you have to use , for whitespace in :main-opts because of reasons... 🙂 )#2018-09-1507:26jcfThat is gnarly, but I love the novelty of your solution.
I'm guessing a solution with similar behaviour to this won't land in tools.deps, but you've inspirited me to think about writing a small library that adds this functionality.
Thanks, @seancorfield!#2018-09-1506:10dottedmag@seancorfield Maybe instead of that one can add a dependency that populates user and make :main-opts require it?#2018-09-1506:13seancorfield@dottedmag I'm sure there are several ways to solve the problem... this was just the first thing that popped into my head 🙂#2018-09-1506:19seancorfieldFeel free to post your solution too...#2018-09-1623:27seancorfield@bhauman Thanks for adding that deps.edn -- see my comments in the now-closed issue... also my comment on the most recent commit (which introduces a syntax error).#2018-09-1623:33seancorfieldIs it a known bug that a local dep inside a git dep doesn't work properly? I expected tools.deps to find the deps.edn inside the cloned repo via the local/root... but it doesn't seem to 😞#2018-09-1623:34alexmillerYes#2018-09-1623:40seancorfieldI couldn't find the JIRA ticket for that @alexmiller -- I assume there is one?#2018-09-1623:51alexmillerThere are several I think - it’s all the local dep path stuff#2018-09-1700:12seancorfieldTDEPS-74 seems to get to the heart of it. OK, I guess I'll go vote for it!#2018-09-1723:46seancorfield@alexmiller @arrdem was offering a patch for the local/root issue I believe...#2018-09-1723:46alexmillerwell I’ve thought about it a lot and haven’t come up with a solution I’m happy with#2018-09-1723:47seancorfieldIf the tools.deps reader converted paths (from :paths, :extra-paths, and :local/root -- and maybe others?) from relative to absolute, based on the location of the deps.edn file in which they occur, would that solve the problem?#2018-09-1723:47alexmillerI don’t think so#2018-09-1723:47seancorfield(or, rephrased, at what point does tools.deps convert paths to absolute ones -- they are absolute in the classpath at the end)#2018-09-1723:48alexmillerthe extension does it when it resolves the paths for a dep#2018-09-1723:49alexmillerbut it’s really about what the “current directory” means when you are resolving a transitive dep by relative path - currently nothing is shifting that context so you always have the root dep context#2018-09-1723:52seancorfieldAh, and paths can be in both the system deps and the per-user deps and of course you don't want those resolved against the file they're defined in... yeah...#2018-09-1723:53seancorfieldAnd by the time you've collapsed all the deps down, you've lost that context, right?#2018-09-1723:56alexmilleryeah. we actually conceived local deps primarily as working primarily with absolute paths - all the stuff people are doing with relative paths and monorepos is somewhat emergent#2018-09-1723:59arrdemSo for https://github.com/fundingcircle/lein-modules, we added a “//” notation somewhat stolen from pantsbuild - the idea was that paths which were not “//” prefixed were treated as relative and just passed around, “//” prefixed paths were treated as being absolute with respect to the project.clj the path occurred in - for us implicitly the “root” project.clj of the repo which emulates the meaning of “//” in Google Blaze derived systems where “//” is repo root when referring to build target names.#2018-09-1802:56alexmillerYeah, I’m not doing that :)#2018-09-1805:21arrdem😉 #2018-09-1805:35arrdemPerhaps a more serious option here would be #file EDN extensions... maybe not suitable for general purpose but perhaps useful for us.#2018-09-1805:42alexmillerNone of that seems necessary to me#2018-09-1805:43alexmillerI think the intent of this kind of setup is clear as is, the questions are in the implementation#2018-09-1723:59arrdemThat’s worked out OK at best for us.#2018-09-1800:05arrdemfile me under monorepo user probably pushing the boundaries of expected behavior.#2018-09-1800:05arrdemI’m currently exploring replacing lein-modules for our mid-size monorepo for a couple reasons.#2018-09-1801:14richiardiandreacouldn't we have a placeholder deps.edn in the root of a monorepo and subdeps consider that first if present for :local/root?#2018-09-1804:55dominicmI did wonder if relative paths would be a different key. #2018-09-1817:54arrdemChatted about this with a coworker some and I think we’re gonna take a swing at adding a resolver type which patches up paths, everything else seeming pretty janky in comparison. We’ll see how that goes.#2018-09-1818:07alexmillerwhat do you mean by a resolver type?#2018-09-1818:14arrdemExtension / coordinate type.#2018-09-1819:06alexmillerI feel like you’re going down a path that is a waste of time#2018-09-1819:07alexmillerWhich you’re welcome to do, but there is no support for installing your own extensions in clj, so you’ll only be able to use this with some frankensetup#2018-09-1819:11dominicmWill there ever be support for custom extensions? #2018-09-1819:30alexmillerundecided#2018-09-1819:31alexmillerI mean, you can use them now if you use tools.deps.alpha as a library#2018-09-1819:31alexmillerit’s just yet another layer of installation complexity when it comes to clj#2018-09-1820:20dominicmI suppose it would require t.d.a to orchestrate itself in order to run again with additional dependencies. Probably not completely horrible if the context allows add-lib. #2018-09-2214:40raymcdermott@alexmiller just checking that the add-lib method from earlier SHAs should still work with 1.10-alpha7#2018-09-2214:41raymcdermottcos I am having fails#2018-09-2214:44alexmillerHaven’t tried it, but not sure why it wouldn’t off the top of my head#2018-09-2214:44alexmillerFails how?#2018-09-2214:45raymcdermottgives back a false#2018-09-2214:45raymcdermottfor the example memoize (and others)#2018-09-2214:46raymcdermottcould be something on my end - just wanted to make sure that it wasn’t meant to be deprecated#2018-09-2214:49alexmiller$ clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha7"} org.clojure/tools.deps.alpha {:git/url "" :sha "d492e97259c013ba401c5238842cd3445839d020"}}}'
Downloading: org/clojure/clojure/1.10.0-alpha7/clojure-1.10.0-alpha7.pom from
Downloading: org/clojure/clojure/1.10.0-alpha7/clojure-1.10.0-alpha7.jar from
Clojure 1.10.0-alpha7
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"})
true
user=> (require 'clojure.core.memoize)
nil#2018-09-2214:50raymcdermottmaybe false is OK if the lib is already available#2018-09-2214:50alexmilleryeah, you’ll get true only if added a new lib(s)#2018-09-2214:50alexmillerhttps://github.com/clojure/tools.deps.alpha/blob/add-lib/src/main/clojure/clojure/tools/deps/alpha/repl.clj#L60#2018-09-2214:52raymcdermottok, great - thanks for checking#2018-09-2214:55alexmillerit can’t be deprecated - I haven’t even released it yet :)#2018-09-2214:58raymcdermottLOL fair point#2018-09-2214:58raymcdermottare you close to release or is the debate still ‘raging’?#2018-09-2215:00alexmillerbeen focused on other things :)#2018-09-2215:02raymcdermottwhat could possibly be more important#2018-09-2215:58raymcdermottthe problem I’m having might be related to Cursive as the tests all pass via the CLI#2018-09-2215:58raymcdermottand fail via the deps.edn default REPL there, so I’ll mention it on that channel later#2018-09-2408:35martinklepschAre there any known issues around depending on an artifact like this: https://repo.maven.apache.org/maven2/ml/dmlc/xgboost-jvm/0.72/#2018-09-2408:35martinklepschi.e. the only jar for the version has a -javadoc classifier #2018-09-2412:06alexmillerWhat are you expecting to happen?#2018-09-2412:06alexmillerWhat is your deps.edn?#2018-09-3008:07alanHi, sorry for the late response, I raised the issue because while I can build without any issue with leiningen I couldn't make the cljdoc build (which uses tools.deps) work#2018-09-3008:11alanThis is the tools.deps {:deps {org.clojure/clojure #:mvn{:version "1.9.0"}, ml.dmlc/xgboost4j-example #:mvn{:version "0.72"}, ml.dmlc/xgboost-jvm #:mvn{:version "0.72"}, org.clojure/clojurescript #:mvn{:version "1.10.339"}, org.clojure/tools.namespace #:mvn{:version "0.2.11"}, clj-boost/clj-boost #:mvn{:version "0.0.1"}, org.clojure/java.classpath #:mvn{:version "0.2.2"}, org.clojure/data.csv #:mvn{:version "0.1.4"}, ml.dmlc/xgboost4j #:mvn{:version "0.72"}, codox/codox {:exclusions [enlive hiccup org.pegdown/pegdown], :git/url "", :sha "4b0720941083fda9643d905f0854fabea55b175f", :deps/root "codox/"}}}#2018-09-3008:13alanIt seems impossible to get the XGBoost stuff in this way. While in leiningen I have [ml.dmlc/xgboost4j "0.72" :extension "pom"] and everything's fine#2018-09-3012:51alexmillerpom extension is not supported - what are you expecting that to do?#2018-09-3013:18alanI'm not using it, is in cljdoc to build the package, but now I guess there's not much it can be done about it#2018-09-3014:03alexmillerSorry, I’m not understanding the problem. It would be very helpful to have a description of the problem, with the things needed to reproduce it, what you expect to happen, and what is actually happening. I feel like the above has half of that but it’s very confusing to me what you’re doing.#2018-09-3014:07alanI have a dependency, the only way to get it is to use the :extension "pom" flag, I develop, build and release my library with lein, no issues. I wanted to build docs with cljdoc which uses tools.deps and the build fails because of that dependency#2018-09-3014:21alexmiller“the only way to get it is to use the :extension "pom" flag” - why?#2018-09-3014:23alanhttps://mvnrepository.com/artifact/ml.dmlc/xgboost-jvm/0.80#2018-09-3014:25martinklepsch@alexmiller because the dependency doesn’t come with a regular .jar file — I don’t know too much about why you package libraries like this but it seems to be a thing in this case
And as far as I understand nobody (using tools.deps) will be able to use @U7P0DPQ1H s library which depends on this artifact #2018-09-3014:37alexmillerIf it doesn’t have a jar, then i don’t get how it affects the classpath. #2018-09-3014:39alexmillerI guess maybe by having deps#2018-09-3014:40alexmillerWhich I would actually expect to work#2018-09-3014:41alexmillerIf someone could file a jira in TDEPS about this, I will look at it more closely later this week#2018-09-3014:56martinklepschYeah would assume that the point of this kind of artifact is usually its deps, I.e. it’s a kind of Uber-module #2018-09-3014:57martinklepschwont have time to file a ticket in the next few days...#2018-10-0415:39alan@alexmiller I just opened an issue https://dev.clojure.org/jira/browse/TDEPS-98 hope it's ok#2018-09-2410:28mkvlrMy issue https://dev.clojure.org/jira/browse/TDEPS-50 still reproduces after it has been marked as a duplicate of https://dev.clojure.org/jira/browse/TDEPS-12 which has been marked as fixed. Can someone with access please reopen this for me?#2018-09-2412:04alexmillerAre you using the latest release?#2018-09-2412:05alexmillerAnd what is your deps.edn? There is some new syntax from TDEPS-12 with an example in the comments there.#2018-09-2412:09alexmillerSpecifically you’ll want the lib just once with a coordinate that has :classifier [“” “native”]#2018-09-2412:13alexmiller{:deps {com.github.jnr/ffi {:mvn/version “1.2.16” :classifier [“” “native”]}}}#2018-09-2412:27bherrmannAm I nutty or does the example "hello.clj" example from https://clojure.org/guides/deps_and_cli not work until you add :paths ["."] to deps.edn ??#2018-09-2412:28dominicm@bherrmann that surprises me for two reasons, one is that it should be "src", the other is that it's a default. #2018-09-2412:29dominicmWhere did you put hello.clj?#2018-09-2412:35bherrmannok, yea. hello.clj just needs to be under "src"#2018-09-2412:35bherrmannmy bad... I'm automating a manual process (converting and excel file) and was thinking more script minded, less software development minded (where a src sub directory is more assumed)#2018-09-2414:06alexmiller@mkvlr I looked at this and have reopened - you’re right that this is related to TDEPS-12 but needs a bit more work#2018-09-2414:06alexmillerbut I’m not going to get to that this week#2018-09-2414:09mkvlr@alexmiller thanks a lot! Yes, we can work around by manually specifying it for now so no rush from us. It’s just missing to pick up the classifier when reading the pom if I understand it correctly.#2018-09-2414:13bhaumanI’m writing some docs on resolving dependency conflicts, is there a tools deps equivalent to lein pedantic?#2018-09-2414:15alexmilleryes#2018-09-2414:15alexmillerhttps://github.com/Olical/depot#2018-09-2414:33bhaumanthanks @alexmiller, while that works and is good to know about it implements lein-ancient it doesn’t find dependency conflicts like pedantic 🙂#2018-09-2414:33alexmilleroh, sorry. then no :)#2018-09-2414:34bhauman-Stree helps of course#2018-09-2414:34alexmillerthere is a totally unsupported subject to removal and alteration verbose mode for tools.deps that is primarily designed to be helpful to me#2018-09-2414:35bhaumanoh I’ll put that in the docs then 😉#2018-09-2414:35alexmillerdefinitely an area where more could be done#2018-09-2414:36bhaumanI moved figwheel to Jetty thinking it was a smart move to get https support and to play nicely with larger environment#2018-09-2414:36bhaumanbut dang its prone to tricky conflicts#2018-09-2414:37bhaumanthe way the package is divided up#2018-09-2414:37bhaumanwhen I say tricky I mean tricky for newcomers#2018-09-2414:38bhaumanthe conflict between ring and datomic cloud is the main problem#2018-09-2416:22lilactownyeah I spent about 3 hrs early on trying to get Jetty (for local dev) and datomic cloud working. I switched to http-kit and it is working well so far#2018-09-2521:55bhauman@U4YGF4NGM I’ve got docs on this now#2018-09-2521:56bhaumanhttps://figwheel.org/docs/jetty_conflicts.html#2018-09-2521:56bhaumanpublished it this weekend#2018-09-2521:56bhaumansorry about that#2018-09-2521:57lilactownawesome! ty!#2018-09-2421:05seancorfieldIf I have some.group/artifact {:mvn/version nil} and an :override-deps entry of some.group/artifact {:mvn/version "1.2.3" :exclusions [foo/bar com.quux/blah]} should I expect the :exclusions from the override to be applied to the main dep?#2018-09-2421:05seancorfield(and what happens if there are :exclusions on both the main dep and the override?)#2018-09-2421:05alexmillerthe override overrides#2018-09-2421:06alexmilleras in replace the original (including its exclusions)#2018-09-2421:06seancorfieldCool. So I can rely on specifying :exclusions just once in the overrides -- that's excellent!#2018-09-2707:07tianshuwhen try to uberjar, meet this error
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.quartz-scheduler:quartz-parent:pom:2.3.0 in terracotta-snapshots ()
This dependency, will be handled correct when using leiningen. I don't know how to investigate the issue, where should I start?#2018-09-2712:21dominicmThis looks like TDEPS-50 also?#2018-09-2713:44orestisIs there a tools.deps friendly way to run Eastwood from the command line?#2018-09-2714:35orestisAlso, what is a no-fuss way to build an uberjar for a deps.edn project? I tried pack.alpha, depstar so far and I get weird errors.#2018-09-2714:36orestisdepstar: I get this issue: https://github.com/healthfinch/depstar/issues/5, too much fuss to do manifests etc.#2018-09-2714:37orestispack.alpha: NOTE: /alpha/bootstrap/onejar/src/com/simontuffs/onejar/JarClassLoader.java, line -1: Some input files use unchecked or unsafe operations.
NOTE: /alpha/bootstrap/onejar/src/com/simontuffs/onejar/JarClassLoader.java, line -1: Recompile with -Xlint:unchecked for details.
~/d/n/c/october (master|✚1…) $ java -jar output.jar
Exception in thread "main" java.lang.ClassNotFoundException: clojure.main
at com.simontuffs.onejar.JarClassLoader.findClass(JarClassLoader.java:713)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at com.simontuffs.onejar.JarClassLoader.loadClass(JarClassLoader.java:630)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.simontuffs.onejar.Boot.run(Boot.java:336)
at com.simontuffs.onejar.Boot.main(Boot.java:168)
#2018-09-2716:37seancorfield@orestis For pack.alpha, that seems to indicate it didn't include the Clojure runtime -- no idea about that since I haven't tried it but @U09LZR36F (the author) is very helpful. For depstar, adding the manifest manually is an easy step and I like the raw simplicity of it. Haven't tried cambada so thanks for finding that and posting the link!#2018-09-2716:38orestisIt’s up on the tools.deps wiki and has a very good README :)#2018-09-2716:39seancorfieldYeah, just looking it over. Very nice!#2018-09-2716:59dominicmI think the pack bug would be fixed by using the latest version. #2018-09-2716:59dominicmI haven't updated the installer, and I should! #2018-09-2714:40orestisHooray - https://github.com/luchiniatwork/cambada seems to work out of the box, with the only addition being an extra command-line parameter to specify the main namespace#2018-09-2716:32andy.fingerhutI do not have a tools.deps friendly way handy and worked out for running Eastwood from the command line, but if you look at this section of the Eastwood docs on how you can do it from the REPL, without Leiningen involved, you or someone better at tools.deps knowledge than I might be able to figure it out. I'd be happy to add a note like that to the Eastwood README if it isn't too difficult to explain: https://github.com/jonase/eastwood#running-eastwood-in-a-repl#2018-09-2716:33andy.fingerhutWarning: The latest Eastwood 0.2.9 release has a bug in which when you try to use it, it will download (if you don't already have it) and use Eastwood 0.2.8 instead. You may want to simply specify Eastwood 0.2.8 for now, since that is what you will end up using anyway.#2018-09-2716:47seancorfield@orestis clj -Sdeps '{:deps {jonase/eastwood {:mvn/version "0.2.8"}}}' -e "(require '[eastwood.lint :as e])" -e '(e/eastwood {:source-paths ["src"] :test-paths ["test"]})'#2018-09-2716:48seancorfieldYou could squirrel most of that away in your ~/.clojure/deps.edn file as an alias...#2018-09-2716:51seancorfield:eastwood {:extra-deps {jonase/eastwood {:mvn/version "0.2.8"}}
:main-opts ["-e" "(require,'[eastwood.lint,:as,e])"
"-e" "(e/eastwood,{:source-paths,[\"src\"],:test-paths,[\"test\"]})"]}
(note that you need , instead of space!)#2018-09-2718:09orestisI ended up doing this: :eastwood {:extra-deps {jonase/eastwood {:mvn/version "0.2.8"}}
:main-opts ["-m" "eastwood.runner"]
:extra-paths ["test" "tools"]}
And I made a tools directory to contain simple namespaces like that. In this case: (ns eastwood.runner
(:require [eastwood.lint :as e]))
(defn -main [& args]
(e/eastwood {:source-paths ["src"] :test-paths ["test"]}))#2018-09-2718:55seancorfieldOh, that's nice! 👍:skin-tone-2:#2018-09-2717:01orestisYeah, I saw that trick on your dot-Clojure and was about to ask! Is that a known bug?#2018-09-2717:02orestisI guess though I can just as easily make a tiny namespace to do this. #2018-09-2717:07seancorfield@orestis It's to do with how the strings are run through the clojure shell script and into the main opts cache file and then read back. So, yeah, it's a "known feature" 🙂#2018-09-2717:07orestisRight, so it’s not going away any time soon I guess?#2018-09-2717:08orestisI remember some mentions of caching compiled dependencies to speed up startup time - any ideas about that?#2018-09-2717:10orestisMore questions - in deps.edn, can you actually override the entire top-level deps entry? Or can you only append?#2018-09-2717:12orestisRationale: most of my aliases so far have to do with running lint tools, packaging, test runners etc. These namespaces have no need to access my actual app’s classpath. I guess there has bound to be some overhead when the class path is unnecessarily large?#2018-09-2717:13orestisI know that clj will cache the classpath and it’s actually very noticeable. #2018-09-2904:07miridiusHi all, what's the best practice for running a deps.edn based project in production? Is it ok to run it "dynamically" with clojure -Aprod -m my.app.main or is it better to AOT compile the app -- and if so, how?#2018-09-2906:06seancorfieldAOT really only saves you startup time.#2018-09-2906:06dominicmWe do that sometimes. No complaints. #2018-09-2906:06seancorfieldWe still use JAR files for production deployment but we never AOT anything.#2018-09-2906:07seancorfieldWe use uberjars for the convenience of knowing our production system will never be dependent on reaching out to Maven or Clojars at application startup. But we use clojure.main as our Main-Class.#2018-09-2906:10miridiuswhat do you use for creating the uberjar? juxt/pack.alpha -> Capsule?#2018-09-2906:10seancorfieldWe're still using Boot but we're moving toward clj/`tools.deps`.#2018-09-2906:11seancorfieldI'm leaning toward depstar for the simplicity of the tooling, but cambada seems to be very slick.#2018-09-2906:11miridiusah ok 🙂 thanks. I'm currently using lein for the uberjar task and literally nothing else#2018-09-2906:13seancorfieldWe outgrew Leiningen years ago and switched to Boot -- we've built almost our entire build/test/deploy pipeline on top of that.#2018-09-2906:13seancorfieldBut we're in the process of retooling to something simpler, based on deps.edn etc.#2018-09-2906:13miridiusdepstar and cambada both look really cool! thanks for those#2018-09-2906:14miridiusYeah I've tried both leiningen and boot and found them both to increase the amount of complexity I face rather than decrease it. Troubleshooting them when they don't behave is especially painful! So I've happily moved everything to tools.deps, and the only piece missing now is production deployment#2018-09-2906:15seancorfieldOur build.boot file is close to 2,000 lines. We have a ways to go 🙂#2018-09-2906:15miridiuswow...#2018-09-2906:17miridiusI use docker so theoretically don't have any need to uberjar anything, as long as I can make sure all the deps are there. I was thinking of adding a :quit command-line arg to my main class that quits immediately and just running that at build time so that it downloads all the deps. Kind of hacky though...#2018-09-2906:18seancorfieldCan't you do clj -Stree to force the downloads?#2018-09-2906:20miridiusoh! I didn't know it would download the deps, I thought it just prints them#2018-09-2906:20miridiushey btw, I use your dot-clojure repo, it's really helpful 🙂#2018-09-2906:21miridius-Stree works... nice#2018-09-2908:15dominicmWe (JUXT) use pack in many projects now, it's gone very well overall. #2018-09-2906:07seancorfield^ @miridius#2018-09-2906:09miridiusthanks for the tips 🙂#2018-09-2921:15bbrinckHi, I just made a small utility (using clj-new- thx @seancorfield) to validate deps EDN from stdin - https://github.com/bhb/check-deps. Feedback/questions welcome 🙂#2018-09-3002:12richiardiandreaNice!#2018-09-2921:17bbrinck(and suggestions)#2018-09-3012:47krukowRunning into an issue migrating from lein to tools deps. REPL works, but when I try to uberjar the project I get the following error:
Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
https://gist.github.com/krukow/b2b6ef6536cb8cf07cc8f613afaa8108
Would someone here be kind enough to help be figure out what's going on?#2018-09-3012:51krukowHmmm - resolved by cleaning .m2 and adding
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2018-09-3012:51krukowdid this mean I had a corrupt .m2#2018-09-3012:55alexmillerThose repos are already included by default so shouldn’t need those#2018-09-3012:55alexmillerBut maybe a bad m2#2018-09-3012:58krukowthanks Alex - I'll remove and simplify, I think I'll clean the entire .m2 and re-build#2018-09-3012:58krukowmigration from lein to deps was very easy 🙂 thanks!#2018-09-3013:00krukowwait - this is weird ... I'll double check but what I'm seeing now is#2018-09-3013:02krukowremoving :mvn/repos from deps.edn => results in the above ArtifactDescriptorException: exception#2018-09-3013:03krukowadding
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
=> uberjar builds#2018-09-3013:07krukownot happy about using http though#2018-09-3013:09krukowException in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:323)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:192)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:253)
at clojure.tools.deps.alpha.extensions.maven$eval2646$fn__2648.invoke(maven.clj:79)
at clojure.lang.MultiFn.invoke(MultiFn.java:243)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:170)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:152)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:215)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
at cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109)
#2018-09-3013:11krukowthis is independent of the state of the .m2 dir#2018-09-3013:12krukowseems that this call cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109) => clojure.tools.deps.alpha$resolve_deps behaves differently depending on whether I have
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
or not in deps.edn#2018-09-3013:12krukowstrange#2018-09-3020:25seancorfield@krukow I just tried with your deps.edn and that build command and it worked just fine. I got this warning tho' Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method, or the namespace has not been AOT-compiled.
#2018-10-0114:14krukow@seancorfield thanks for helping out - I ended up with this uberjar command:
:uberjar
{:extra-deps
{luchiniatwork/cambada {:mvn/version "1.0.0"}}
:main-opts ["-m" "cambada.uberjar"
"-m" "vsts-flow-metrics.cli"]}}
which should help for the main warning.
Then without specifying a :mvn/repos in deps.edn I get:
Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:323)
...
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
at cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109)
But If deps.edn contain
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
the command completes#2018-10-0114:14krukowI'll see if I can create a minimal repro#2018-10-0114:14krukowand I use clj -A:uberjar -Srepro -Sforce to create the uberjar#2018-10-0114:46timgilbertSay, using an s3p private repo I get a lot of messages like these when starting clojure:
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Apparently this file is just missing, but I definitely don't have it in my private repo. Is there any way to tell clojure and/or maven to only use the private repo for artifacts from a specific namespace?#2018-10-0116:05timgilbertAlternately, is there a way to suppress that message from appearing? It's not so good for shell-script type tasks#2018-10-0116:08timgilbertThe full panoply of messages is more like this:
Downloading: com/google/code/findbugs/jsr305/maven-metadata.xml from
Downloading: com/google/guava/guava/maven-metadata.xml from
Downloading: commons-codec/commons-codec/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-kinesis/maven-metadata.xml from
Downloading: com/amazonaws/amazon-kinesis-client/maven-metadata.xml from
#2018-10-0116:20timgilbertI guess what I'd like is the ability to set the <updatePolicy> value on my S3 repository to never, which I think would make this behavior go away. https://maven.apache.org/pom.html#Repositories#2018-10-0119:08zane:jvm-opts isn't a valid top-level key, right? Only inside an alias in :aliases?#2018-10-0119:43alexmillercorrect#2018-10-0119:43alexmillerthere is a ticket wrt default jvm-opts#2018-10-0120:31zaneOkay. Thanks, Alex!#2018-10-0205:45kommenis there a way to figure out in a clojure process started with clj which aliases were used to start it?#2018-10-0206:10seancorfield@kommen No.#2018-10-0206:11seancorfieldThe aliases are only used to build the classpath (and main opts etc). By the time your program is running, none of that is available.#2018-10-0206:11seancorfieldWhat is the problem you're trying to solve here?#2018-10-0206:11kommenI’m looking into a solution for this cider issue: https://github.com/clojure-emacs/cider/issues/2308#issuecomment-426029865#2018-10-0206:13kommencider checks if dependencies for certain functionality are on the classpath. until now, simply by splitting the classpath entries and interpreting that. this falls apart when using tools.deps git and local coordinates#2018-10-0206:15seancorfieldYup. Well, I think CIDER is trying to be far too clever here and making all sorts of unwarranted assumptions -- but it sounds like Bozihdar is aware of the problem and will figure it out -- he's been doing amazing stuff with CIDER...#2018-10-0206:16seancorfieldclj and tools.deps are making a lot of people reconsider their assumptions 🙂#2018-10-0206:23kommenok, thanks#2018-10-0206:49dominicmThe safe way to do it is to check for the existence of namespaces/classes, not to search the classpath for them.#2018-10-0220:13krukow@seancorfield @alexmiller I have a simple repro (on my machine) - from a clean install. Unzip the .zip file into an empty directory and run clj -A:uberjar -Srepro -Sforce - would any of you be so kind as to confirm / invalidate the repro so I can make a formal issue?#2018-10-0220:15krukowFYI the repro sequence was: with an empty /.m2, /.clojure on OS X high sierra with brew install clojure, `clj -Sdeps '{:deps
{seancorfield/clj-new
{:git/url "https://github.com/seancorfield/clj-new"
:sha "21ca1b27f46dc324be084ba839beca555aeda387"}}}' \
-m clj-new.create \
app \
krukow/artifact-exception` and then clj -A:uberjar -Srepro -Sforce in the folder where simple-repro.zip is unzipped.#2018-10-0220:18krukowthen add
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2018-10-0220:18krukowto deps.edn and things work#2018-10-0220:20krukowFYI: posting this to https://github.com/luchiniatwork/cambada/issues#2018-10-0220:34dominicm@krukow this makes sense to me. Cambada doesn't consider the system deps.edn file. #2018-10-0220:36seancorfieldOuch! That's quite the gotcha!#2018-10-0220:43krukow@dominicm I think that's correct! I'll add to: https://github.com/luchiniatwork/cambada/issues/15#2018-10-0220:43krukowthank you!#2018-10-0220:44krukowbut the workaround is easy then - I should have thought of this 😮#2018-10-0220:44krukowOne more general question: where can I find the source code for the clojure & clj CLI tools#2018-10-0220:46seancorfieldThe brew-install and tools.deps.alpha repos under the clojure org.#2018-10-0220:46seancorfield(the former is the shell scripts etc, the latter is the library they use for all the dependency stuff)#2018-10-0220:47krukow👍 thank you Sean#2018-10-0222:41amar@krukow this PR should fix the issue you were having https://github.com/luchiniatwork/cambada/pull/14#2018-10-0305:09dominicmAlso means it doesn't need to merge in paths#2018-10-0305:29zaneIs it just me or is the -v / --var option for com.cognitect/test-runner broken? Setting it appears to trigger a crash because set is being called on a single-argument call to map, which returns a transducer: https://github.com/cognitect-labs/test-runner/blob/master/src/cognitect/test_runner.clj#L20...L22#2018-10-0305:29zaneHoping to get one more set of eyes on this before I file an issue.#2018-10-0305:46seancorfieldIt's missing var as the second argument to the map call. Def. a bug.#2018-10-0313:51zaneThat was my guess too. Thanks!#2018-10-0307:06krukow👍 amar!#2018-10-0314:49hlolliI added this into my deps.edn :jvm-opts ["-Xms2G" "-Xmx5G"] started repl with cider, but I can't in any way see if these options are present. This is not present in the command line args if I look at the java process in my system monitor.#2018-10-0315:06ghadiI don't think these take effect except within an alias#2018-10-0315:13hlollioh ok, good to know 🙂 @ghadi#2018-10-0315:26orestis@hlolli see in #cider for a way to start cider with a specific alias. #2018-10-0315:26orestishttps://clojurians.slack.com/archives/C0617A8PQ/p1538572945000100#2018-10-0316:03hlollibingo works, I've been bit turned off by dir-locals due to "unsafe" popup. But the popup didn't appear this time.#2018-10-0406:36dominicmI think I understand why, but should it be the case that, if an alias specifies main-opts (mine has -m in), then -e and -i are ignored? The output of clj --help doesn't exactly suggest that -e and -m are mutually exclusive, especially as they are "different".
Currently cider tries to use -e in order to start nrepl without interrupting the users choice of main-opts.#2018-10-0406:39dominicmOne solution might be to add a java class to cider releases which triggers in response to a jvm option, but that's not going to be my first approach if I can help it. -e is just lovely as a general tool.#2018-10-0412:36genekimNot sure where I should ask this, but wanted to put this out there —
After listening to a @apropos_cast, I took the plunge and tried clj and tools.deps.
I was blown away by how easy it was! I used lein-tools-deps by @rickmoynihan — it took only 10m to transition my first project over, and was able to quickly do three other lein projects. It was fantastic to be able to share library code w/o publishing to local maven repo. It made using the monorepo pattern fantastically simple.
Marvelous work, Rick — and tools.deps is fantastic!
However, on my laptop, I found that “clj -m foo.main” takes 15s to startup, 2x longer than “lein run”. (Which is not what I expected. I expected startup times to be the same, or lein to be worse, given my understanding that there’s one more JVM being started?)
It’s not in a public repo, so it’d take some time to come up with something that you could reproduce — but if anyone is interested, I can capture some traces, or with some help, I can try to come up with a reproducible test case?#2018-10-0413:06rickmoynihanAlex Miller deserves almost all of the credit. ltd does almost nothing. But thanks for the thanks.#2018-10-0413:52alexmillerOn the timing, was that first time or second time (after classpath is cached)?#2018-10-0414:39genekim@U064X3EF3 brilliant work! It’s a marvel! Thank you!
On timing: I ran all the timings 5-10 times, so I’m presuming this is with classpath caching.#2018-10-0412:42mpenetI think lein has default jvm flags optimised for startup time, clj I don't think does anything like that#2018-10-0412:42mpenetI could be wrong tho#2018-10-0413:52alexmillerI doubt you’ll see that much of a difference from those flags#2018-10-0413:53alexmillerIf you’re using :aot with lein then that could be a big difference#2018-10-0413:53mpenetYeah in general when people complain about startup time I reply that their workflow is prolly not really optimum#2018-10-0413:53alexmillerLein would use the compiled classes #2018-10-0413:54mpenetdoing cli tools with clojure is another matter, solved with other tools probably#2018-10-0415:07bhaumananyone agree? https://github.com/clojure-emacs/cider/issues/2473#2018-10-0416:22cflemingI’m having a problem using tools.deps as a library. If I don’t set the CWD of the process using it to the correct directory, then the manifest type detection is broken.#2018-10-0416:25cflemingIs that expected, a known bug, or something I should file an issue for? I can’t see anything immediately obvious in the tracker.#2018-10-0416:36alexmillerIf you are using absolute paths, then I presume things are fine. If you are using relative paths, then it seems reasonable to expect that they are relative to the current working directory#2018-10-0416:37alexmillerTransitive local deps with relative paths are a problem and that’s a known issue#2018-10-0416:37cflemingI would have assumed that they would be relative to the file in which the ref was found.#2018-10-0416:39cflemingIn this case, I have a root project which has a :local/root of “modules/something”. That deps.edn has another local ref with a relative path, which sounds like it’s a known issue?#2018-10-0416:39alexmillerThat’s the known problem - TDEPS-74 #2018-10-0416:40cflemingI don’t think that’s the problem that I’m seeing, I’m seeing issues with manifest detection there too.#2018-10-0416:41cflemingBut since the manifest detection is relative to the CWD, that will never work.#2018-10-0416:43cflemingI assume it’s all related to the same problem.#2018-10-0416:48cflemingI can’t see a good fix for this. detect-manifest would require the full path to the original file, and the only way I can see to pass that would be to add a new key to local coords when calling manifest-type.#2018-10-0416:49cflemingEven expand-deps doesn’t have that info right now, although I’m assuming that could be passed as a param.#2018-10-0416:50cflemingUgh, in fact, not even resolve-deps has that info, since it’s passed a deps map, not the original files.#2018-10-0416:53alexmillerYes, it’s difficult to fix, which is why it’s still open#2018-10-0416:53alexmillerI have a couple ideas on it to look at though#2018-10-0416:53cflemingIt seems like the best place to fix it would be to absolutise those paths in read-deps.#2018-10-0416:57cflemingIn fact, if that happened as part of canonicalisation in slurp-deps, then that should also work for transitive deps.#2018-10-0416:59alexmillerThe related problem is that for local deps, it can’t build the proper set of deps.edn to merge - they’ve already been merged upstream#2018-10-0417:00alexmillerThis relates to the notion of naming and separating the deps files#2018-10-0417:00cflemingThe idea would be to merge local deps with the system and user deps files too?#2018-10-0417:00alexmillerWell the key is to NOT get the original project deps.edn#2018-10-0417:00cflemingRight.#2018-10-0417:01alexmillerAnd all of this stuff also is related to -Sdescribe and how things are installed on the various platforms#2018-10-0417:02cflemingThe only way to reliably not get that is to pass either the paths or the read maps around, I guess, since merging is destructive.#2018-10-0417:02alexmillerI need some focus time to sit down and wrk through all these things.#2018-10-0417:02cflemingI guess you could merge system and user, and then pass that around and merge it with the other deps files, including the main project one, when you read them.#2018-10-0417:03cflemingThat changes the whole API though, unless you stored that in the map you pass around I guess.#2018-10-0417:03cflemingUnder a ::base-config key or something.#2018-10-0510:00mpenetis there a workaround to https://dev.clojure.org/jira/browse/TDEPS-51 ?#2018-10-0510:01mpenetI guess classpath-overrides is a possibility but that's a bit gross#2018-10-0515:22genekimGot it — thanks for the insights on lein compiles vs. clj, @alexmiller @mpenet !#2018-10-0517:38adamfreyIf I'm making a library with tools.deps and it includes a Java file in it, how can I have that library be consumable in a way that allows people to import the java class? I know it the .java file needs to get compiled into a .class file at some point, which makes sense in an uberjar, but what about pulling in my library via git or :local/root? Would it work / is it advisiable to put the .class file into version control?#2018-10-0518:15alexmillerYou can’t - you should use a build tool to create and deploy an artifact #2018-10-0520:29nwjsmithI’ve been playing around with tools.deps using Maven as a build tool, and it’s been a good experience. clj -Spom -Srepro does mess with my pom.xml in undesirable ways, but testing is straightforward with the clojure-maven-plugin, and building uberjars with maven-shade-plugin works great. The most pleasant thing for me has been the speed relative to Leiningen, it’s much faster.#2018-10-0909:30martinklepschreminds me of this: https://github.com/danielsz/meyvn#2018-10-0520:30nwjsmithI’m not sure why I haven’t seen this set up more often in the wild#2018-10-0520:30nwjsmithFear of XML maybe?#2018-10-0810:34isaacclojure cli don’t care -Dmaven.repo.local?#2018-10-0811:57eval2020@isaac you can add :mvn/local-repo "some/path" to deps.edn.#2018-10-0812:15isaacIs the :mvn/local-repo will be instead of “~/.m2”?#2018-10-0812:25eval2020the provided path will be used instead of ~/.m2/repository#2018-10-0905:00Michael FianoHello all. I just noticed that -A must be supplied before the -r argument for them to take effect. Is that intentional or a bug?#2018-10-0905:01Michael FianoCancel that. I see in the usage notes that is intended#2018-10-0923:35kennyGetting UnknownHostKey: . RSA key fingerprint is when running clj -Spath. Any ideas?#2018-10-0923:37kennyAh needed to add github to known_hosts. That should produce a better error message.#2018-10-1005:43dominicm@kenny the issue is because of ecdsa I believe. #2018-10-1008:39henryw374It's a bit frustrating to have to wait so long for Windows support. https://dev.clojure.org/jira/browse/TDEPS-67 has been around for a while... Please add your vote! (not sure if it'll make a difference but it can't hurt). Thanks#2018-10-1015:39dominicmWe're developing a situation where a few aliases have been split up in the name of DRY. This means that to start the project in "dev mode" requires a difficult number of aliases. Has anyone tried tackling this yet?
I'm currently thinking of writing something that merges together aliases so you can create a "super-alias" that's easier to maintain.#2018-10-1109:11Andreas LiljeqvistWe use makefiles to handle starting tricky combos#2018-10-1109:13dominicmThat doesn't go very well with cider-jack-in though, or any other editor integration?#2018-10-1109:17Andreas LiljeqvistYou are right, we use it more for building releases and such#2018-10-1109:17Andreas LiljeqvistDon't have a good solution for launching dev#2018-10-1015:40dominicmThis is really just a workaround for the lack of ability for aliases to depend on one another.#2018-10-1015:41seancorfield@dominicm Yeah, we're running into that a bit as we convert our Boot-powered monorepo to clj/`deps.edn`...#2018-10-1016:38zaneI'd love to hear more about this if you'd like to share! Why you're doing it, etc. #2018-10-1017:35seancorfield@U050CT4HR We've had our dependencies in external EDN files for ages (years, maybe) and if fact called them deps.edn until clj/`tools.deps` appeared 🙂 I'm running into problems with Boot due to the fileset and pod stuff (I think because our project is large -- 83K lines across 400+ files -- but we haven't managed to tie down a repro case yet). Several of our Boot tasks take a long time to fire up and run, whereas running similar tasks via clj is much faster (since there's a lot less machinery involved).#2018-10-1017:36seancorfieldI'm also a big fan of keeping close to the bleeding edge with Clojure releases and "core" tooling so having fewer dependencies in our system would make me happy.#2018-10-1020:00zaneThat's really helpful Sean. Thanks so much!#2018-10-1015:41dominicm@seancorfield have you had any particular ideas on how to tackle it? I'm quite happy to open source my solution. It will likely be a key piece in edge.#2018-10-1015:42alexmillerhave you considered just repeating yourself? :)#2018-10-1015:45dominicm@alexmiller In some cases, yep! For a single dependency it's usually not so bad. When it's 2 or 3 aliases with 2 or 3 changes, the maintenance burden of bumping in multiple places is failing quite often. I'll admit that I'm trying to add in more "easy" onto this design than usual, as I can't impose pull request standards onto all projects.#2018-10-1015:48seancorfieldWe've had to repeat a chunk of our testing context in two separate aliases, which is a bit annoying. The alternative is a longer string of aliases. Right now we have a dummy project containing a deps.edn with mostly aliases. That has a :default aliases which has all our :override-deps where we "pin" versions of libraries. Then we have a deps.edn in each project that specifies the deps for that project (mostly as some/lib {} since the actual version comes from the :default alias) and references to other subprojects as worldsingles/something {:local/root "../something"}. This works pretty well for working within any given subproject and then we'll likely have a couple of shell scripts for running tests across the whole suite of projects.#2018-10-1015:49seancorfieldDoing this has actually highlighted some problematic cross-project dependencies that Boot sort of hid for us, so that's been nice. And of course being able to use a much more lightweight process is always nice 🙂#2018-10-1015:49dominicmheh, I actually just caught myself this second failing to update one of two places. I only caught it because I realized that a :main-opts looked out of place in the alias I had updated.#2018-10-1015:53seancorfieldOne thing I do find is that I'd desperately love to be able to parameterize deps.edn sometimes. For example, we can have an alias that starts an nREPL server to work with a project in an editor, but we can't affect which port it starts on -- that has to be hardcoded in the alias. So I'm thinking we may end up with one or more aliases that bring in (and run) certain small Clojure scripts that exist purely to support dev/test work. That way we could do clj -A:some:stuff:here <port-number> and run a -main in a dev namespace that starts nREPL on that port.#2018-10-1015:53seancorfieldI'd be interested to hear more about how Cognitect folks deal with this in large projects. hint, hint 🙂#2018-10-1015:53alexmillerdon’t make large projects?#2018-10-1015:54seancorfieldscowl#2018-10-1015:54seancorfieldIs 63K lines of production Clojure and 20K lines of tests "large" tho'?#2018-10-1015:56alexmillerdeps.edn being “just data” (and not having parameters, references, etc) has vast benefits for “simple” and is not something I will give up easily#2018-10-1015:56alexmillerat some point of complexity, you are really doing programming here#2018-10-1015:56seancorfieldOh, true. I totally get the benefits. I just haven't (yet) found the ideal balance yet, with the new tooling.#2018-10-1015:56alexmilleryou can use clj to run programs - so pushing it into a script (which can take args) is one path#2018-10-1015:56dominicm@seancorfield I'm in favor of a main for the nREPL situation.#2018-10-1015:57alexmilleryou can also wrap clj in another program (probably a shell script)#2018-10-1015:57alexmillerbut could also be, say, make#2018-10-1015:58alexmillermake has the nice capability to define dependencies on smaller pieces which I think often is useful in build situations where you are making dependent subtasks#2018-10-1015:58alexmillerditto ant#2018-10-1015:58seancorfieldBeing able to do something like this to run all subprojects tests is convenient enough: for d in * ; do (cd $d; CLJ_CONFIG=../versions clj -A:defaults:test:runner); done
#2018-10-1015:59seancorfieldWe're trying hard to get away from Ant. All that XML shudder#2018-10-1015:59alexmillerI don’t think running a standard “program” per project is any different though#2018-10-1016:00seancorfieldYeah, hence why I'm coming around to having a "dev" project with short Clojure scripts with -main entry points. And, to be honest, that's not hugely different from having tasks in a build.boot file which are all just functions.#2018-10-1016:09dominicm@seancorfield would the dev project be a dependency of all projects?#2018-10-1016:13seancorfieldOnly under an alias for dev/test tooling -- it wouldn't be a regular dependency.#2018-10-1016:19dominicmsorry, yeah.#2018-10-1018:40dominicm@seancorfield once add-lib is stable, it might be a good idea to dynamically fetch dependencies based on the task(s) being run. e.g. nrepl will grab cider/tools.nrepl on demand. That would probably reduce the number of aliases way down. Basically making most of the project static (deps.edn) but providing a dynamism for the non-simple stuff.#2018-10-1615:01sparkofreasonWhat's add-lib? Sounds intriguing.#2018-10-1619:36dominicmIt's a function in a branch of tools.deps#2018-10-1018:42seancorfieldI dream of a world where tooling bootstraps itself on top of prepl instead of needing server-side dependencies 🙂#2018-10-1018:43seancorfield(I love unravel for that reason)#2018-10-1018:48dominicmsocket repl already does that, but I agree 🙂. But I think the same applies (for me) for loading in cljs to do a one-time build.#2018-10-1018:55seancorfieldWell, specifically the tooling. I'd like to see a version of CIDER that could bootstrap itself over a socket REPL or prepl. So you didn't need all the CIDER deps on the project side of the equation. Same with ProtoREPL for Atom.#2018-10-1018:57seancorfieldWhat I like about unravel is that I can spin up a bare Socket REPL (e.g., on a production JAR, via JVM options) and then connect via unravel and when I (require 'compliment.core) it sends it over the wire from the REPL so I don't need it in my project. Then I have a nice interactive REPL with history, documentation hinting, completion...#2018-10-1019:00seancorfieldYour project then becomes tooling neutral. You don't need any dependencies in it to support tooling. And then you can use any tooling with it. I know CIDER can inject itself into a newly-started REPL process, but that's a heavyweight server containing all of those dependencies. I basically want CIDER-like functionality over a bare Socket REPL started from a JAR that doesn't have those dependencies 🙂#2018-10-1019:24dominicmI just discovered I can use find-deps as a filter! Nice. This means I can do !aE in vim, followed by my find-deps command to auto-update the buffer I'm working on. Cool!#2018-10-1019:29dominicm@alexmiller have you had any thoughts about comments being obliterated by -Sresolve-tags?#2018-10-1019:30dominicmI should open a jira if there isn't one#2018-10-1019:30alexmillerreader doesn’t preserve comments#2018-10-1019:31alexmillerI’m not planning on doing any more work on resolve-tags. most likely will actually extract it all out into a separate project and remove it entirely#2018-10-1019:33alexmilleras an external program you could invoke it from clj if needed#2018-10-1019:40dominicmIf it's an external program that shifts expectations I think, cool. I would be more inclined to reimplement it with rewrite-clj so I could preserve comments/formatting.#2018-10-1019:47dominicmI've been thinking about whether a wrapper over rewrite-clj that implements IAssoc is feasible.#2018-10-1020:26alexmillerWell feel free to rewrite it and make it do whatever :)#2018-10-1020:41arrdem@seancorfield have you seen https://github.com/Unrepl/unrepl? It does pretty much that self-bootstrapping dance.#2018-10-1022:11seancorfield@arrdem yes I use that with socket server REPLs a lot. #2018-10-1022:12seancorfieldIt can bootstrap compliment into a running REPL to give you auto-completion, which I like. #2018-10-1022:29andy.fingerhutI guess bootstrap here just means "send the necessary text across the socket to define functions/etc. to create the necessary code on the server?"#2018-10-1022:32seancorfieldYeah. In this case, "send the Compliment library across the socket..."#2018-10-1101:36seancorfieldI needed very simple uberjar functionality with clojure.main as the Main-Class in the manifest. I forked and updated depstar to provide that https://github.com/seancorfield/depstar/commit/b49f15b0fc341f3004cb1bbf037334e3acbcb900 (a PR is in to @ghadi but my last PR to update the readme to explain the lack of manifest hasn't garnered a response yet so...).#2018-10-1101:38seancorfield;; uberjar building:
;; - see for a minimalist approach:
;; - uses my fork that adds manifest support
;; - clj -A:depstar result.jar -m
;; - sets clojure.main-main as the entry point
;; - clj -A:depstar result.jar my.project
;; - sets my.project/-main as the entry point
:depstar {:extra-deps
{com.healthfinch/depstar
{:git/url ""
:sha "a328136da59ea9b83483189451fb71aafd13bdb1"}}
:main-opts ["-m" "hf.depstar.uberjar"]}
#2018-10-1106:35Michael FianoAs a beginner to tools.deps and Clojure in general, I just banged my head against the wall for a couple hours figuring this implementation detail out, because the documentation leaves off this edge case. I discovered something with alias loading I thought would be worthwhile to share here in any case:
Assuming you have 2 aliases with :main-opts keys, regardless if other keys from other alias types are present in them or not:
If -M is specified on the command line for both aliases, only the last alias is used as documented.
If -A is specified on the command line for both aliases, only the last alias is used as per the documentation for -M.
If -A is specified for one, and then -M is specified for the other, only the last one is used.
If -M is specified for one, and then -A is specified for the other, still only the -M one is used, even though -A is later.
It's that last rule that got me good. In short, you have to be very careful here.#2018-10-1108:12martinklepschAnyone recommendations for a thing that will package my deps.edn lib into a jar and install it into the local repo?#2018-10-1108:16martinklepschPack doesn't do regular jars, cambada doesn't install them into local maven, meyvn seems a little more involved#2018-10-1108:33orestisI think the other one is depstar? No idea if it does what you want though.#2018-10-1108:46mpenetyeah they all are a bit tricky to use, each with their own set of tradeoffs#2018-10-1108:48mpenetpersonnaly I am half tempted to just rely on having clj installed and run apps with it, more or less what you do with cljdocs I think @martinklepsch right (bash script that downloads the archive, upacks it and runs)?#2018-10-1108:51mpenetcapsule based build seemed quite a good idea (one of the options on juxt.pack), but I am not sure where that project is heading#2018-10-1109:02dominicm@mpenet happy to answer questions you might have!#2018-10-1109:05martinklepschI'm actually referring to regular jars (not uberjars)... e.g. when publishing a library#2018-10-1109:13dominicm@martinklepsch fwiw, you could use pack with skinny jar mode and write a pom.xml. But you'd have to install that jar yourself (with maven I guess?)#2018-10-1111:25mpenet@dominicm nothing special in mind. pack is imho the most complete solution so far. One minor missing piece would be the ability to specify a main on the one-jar option, but i can live without it#2018-10-1116:24dominicmThere should be an issue open for this. It is something I want to fix. I don't want to require AOT from users, so I want to patch onejar a little to support a default param to clojure.main.#2018-10-1112:23orestisWould be nice if there was only one well supported solution for packaging. I stared with cambada but had to fork to apply some PRs#2018-10-1114:50seancorfield@orestis sounds like me forking depstar to apply a PR to add a minimal manifest 😀#2018-10-1114:51seancorfield@martinklepsch is there a reason you need a jar in your local repo? Can't you refer to that local library source via :local/root?#2018-10-1115:07martinklepsch@seancorfield I can but I have some wierd interdependencies that aren't supported fully by Cursive/tools.deps#2018-10-1115:08cflemingRight, transient :local/root deps are not well supported (by deps itself, and also Cursive - I’m fixing that right now)#2018-10-1115:12orestisEven though I don’t use Clojure professionally yet, I’d chip in for an Arachne-like Kickstarter, but for a new build tool based on tools.deps and simple libraries. #2018-10-1115:13orestisThere’s so much duplication of effort these days - jar building, linters, formatters and so on. #2018-10-1115:15orestisI have a dream for a build tool that would just provide an (overridable/extendable) set of library decisions and some glue code to invoke them from the command line. #2018-10-1115:28Andreas LiljeqvistWhat you are describing is probably a program 😕#2018-10-1115:31Andreas LiljeqvistBut yeah, there is a lot of churn in the dev space right now#2018-10-1115:31Andreas LiljeqvistWill probably be a while before it simmers down with some kind of best practices#2018-10-1115:33orestisThe whole point is to rally everyone around a single tool. I think this has multiplier benefits for the community and newcomers. See for example Elixir and their mix tool. #2018-10-1115:40Andreas LiljeqvistThere is certainly value for newcomers.#2018-10-1115:40Andreas LiljeqvistAtm Lein is probably good for them#2018-10-1115:41Andreas LiljeqvistBut tools-deps is the first system that let me understand wtf was going on when starting up#2018-10-1115:41Andreas Liljeqvistsooo..#2018-10-1115:43orestisYeah, I’m all for tools.deps for dependencies and classpath stuff. #2018-10-1116:28seancorfieldI've been recommending clj to newcomers -- with the caveat that most books/tutorials use lein. At least with clj, you can point newcomers at "official" documentation in a central place and you know the tooling is both simple and well-supported.#2018-10-1116:34orestisAs a relative newcomer, clj is just so much less cognitive overhead than Lein. #2018-10-1115:34seancorfieldFor a long time, at work we've done no AOT and we build over uberjars to have clojure.main as an entry point. Our needs are very simple. We used to run a lot of stuff from source directly via lein and later boot but I think packaging as an uberjar for java -jar is simpler and safer (no need to fetch dependencies onto the target systems).#2018-10-1115:38Andreas Liljeqvist+1 for building uberjars#2018-10-1115:35seancorfieldClojureScript has had years of churn but seems to be settling down now. Maybe Clojure needs a bit of churn to shake things out and give us better, simpler tools too?#2018-10-1115:36Andreas Liljeqvisttools-deps is a great step in that direction. But causes a bit of chaos in the transition#2018-10-1115:46orestisTo be clear, IMO tools-deps enables the creation of such a mostly-glue tool. Otherwise making a new build tool from scratch is a huge undertaking. #2018-10-1116:05arrdem@martinklepsch the official story there seems to be write a pom.xml yourself, use the -Spom task to keep the dep lists in sync? (I don’t really understand what it does) and mvn install yourself.#2018-10-1116:26dominicmHaving attempted to write that build tool, I'm not happy with anything I've come up with. Build tooling is hard to get right, especially around tools like figwheel. #2018-10-1116:35orestisShare notes perhaps? Any attempt to explore the space is very valuable.#2018-10-1116:36dominicmKrei.alpha and kick.alpha are the two attempts by JUXT. We also did mach in the past which we used for alternative purposes. #2018-10-1116:38dominicm@U064X3EF3 keeps mentioning that builds are programs, so I suspect that both of those attempts to be data driven are fatally flawed.
I've been thinking of implement a paper by peyton jones for creating build systems a la carte, maybe that will unlock the pieces needed, but I'm not sure yet. #2018-10-1116:55orestisMy naive attempt so far is to just invoke some namespaces - either external or my own via aliases. My need is for more well rounded and robust libraries to do things like jars and so on. #2018-10-1116:57dominicmCaching. Minimal builds. Early cut off. These are the general problems to solve. Cloud building is a nice to have in some contexts, but none I'm in. #2018-10-1117:01orestisI’ll look at krei and kick, thanks. #2018-10-1117:05orestisI don’t think that there is any way around the current slow startup of clojure, is it? Even with precompiled stuff, you will get at least a handful of seconds which is already too much. Hard stuff. #2018-10-1117:09rickmoynihanyou could always fire up a build-tooling repl once, that puts you in a namespace with functions like (build) (test) (do-everything)#2018-10-1117:16zane@orestis Have you seen the results people are getting with GraalVM?#2018-10-1308:54dominicmSomething quite hard to build at a REPL is AOT-compiling namespaces. You have to start a new JVM to do that. You might be able to get away with something like a pod, but that's the best I have. Just as a result of how it decides to cache/skip namespaces.#2018-10-1116:31lilactownespecially since the UX is so dependent on perf. everyone wants things to be fast#2018-10-1116:31lilactownor at least, fail fast#2018-10-1116:39dominicmTo be honest, I think you have to just make it a library that you can run from the repl. It has to happen around the Clojure command line tools, else there is too much friction.#2018-10-1116:42dominicmEven shell scripts get in the way of tooling. Cursive won't work if you require funk.#2018-10-1116:42seancorfieldI really do like depstar's minimalism but it's frustrating that it omitted a minimal manifest -- and I'm not sure why it did that. Everything else in the build space is much more complex 😐#2018-10-1116:44dominicm@seancorfield the easy answer is that generating manifests is out of scope. War is another format that needs a manifest to be written. Simpler to let users decide how to handle extra files in their classpath. #2018-10-1116:44dominicmI don't mean to speak for the author, just a thought I've had working on pack.#2018-10-1117:04rickmoynihanyes, not so minimal when you start having to do that…#2018-10-1117:09seancorfieldAdding a manifest with just a main class is only a few lines of Clojure when you're building a JAR tho'... 🙂#2018-10-1117:28dominicmBut it conflicts with any additional things a user may want to insert into a manifest themselves. Users can also generate a manifest in a few lines of code. In most cases of can just be hardcoded by the user. #2018-10-1118:47seancorfieldRight, which is why it's important that adding the minimal manifest is optional. depstar specifically excludes META-INF/*.MF files from the classpath so the resulting JAR is guaranteed to have no manifest. My fork allows users to conditionally add a minimal manifest that specifies Main-Class (only). That seems to be such a common need -- for java -jar ... -- that baking it in as an option seems worthwhile.#2018-10-1118:48seancorfieldThe code is based on Boot's create-manifest so it could also take a hash map of key/value pairs to add as extra manifest attributes, but that's not exposed to the command-line in my fork (yet).#2018-10-1118:48dominicmI see. That seems quite wrong. It also removes license files which is not allowed under some licenses like Apache#2018-10-1118:49dominicmFun fact: run manifest doesn't capture the full capabilities of manifests! #2018-10-1118:49seancorfieldYeah, I noticed that and thought it was weird. I don't know why it does that (and Ghadi hasn't responded to any of my questions about it yet).#2018-10-1118:51seancorfieldWe're not redistributing our JARs so the license issue isn't a problem -- and depstar filters out a lot of other stuff too so it's pretty brutal in that respect.#2018-10-1118:35arrdemIs there an algorithmic reason beyond wanting to track the path for every expanded dep that expand-deps couldn’t group deps into resolution batches by manifest type? The Maven implementation used to read manifests seems designed for reading many manifests in one go, and less so for reading a single manifest at a time.#2018-10-1121:03arrdemThe hard thing is proving that letting Maven potentially solve range constraints instead of using deps’ behavior for everything can’t produce a different final dependency solution 😕#2018-10-1122:19alexmillerMaven will produce a different solution, and we don’t want it#2018-10-1123:28arrdemAh. Yeah if you have a non-Maven pinning (or even just winning coordinate) for a dep, and Maven resolves something else, you can’t really back the transitive consequences of that out of Maven’s solution so you loose minimality for sure which can have other consequences.#2018-10-1119:11Michael FianoRegarding my message above about aliases, does anyone think this deserves an issue opened to at the very least document this edge case?#2018-10-1119:15arrdemThe order independence of -M winning when -A and -M is present is probably worth documenting.#2018-10-1119:15arrdemSo yes IMO.#2018-10-1119:20dominicmI would prefer to see it fixed, but documentation would be an improvement over the current situation. #2018-10-1119:23Michael FianoPersonally I'd like to see the behavior change. The reason why is this:#2018-10-1119:23Michael Fiano:
I have a :dev alias that is only ever project-local. I use it for example to require my namespace when starting up an nrepl in cider. This requires a :main-opts key, making it of the -M type, but as per the documentation, only the last -M type is ever used. I also want to compose this with a debugging alias to say depend on and require spyscope with another alias. Currently this is impossible without putting that in the :dev alias and thus copying it to be project-local instead of a global alias.#2018-10-1119:24Michael FianoUnless there is some other way to compose a global main-opts with a project local one that I do not know of#2018-10-1119:25Michael FianoMaybe I'm just asking too much from this too new tool?#2018-10-1119:28seancorfieldFWIW, I only ever use -A and I split aliases out so I run into this issue (downside: you need to specify multiple aliases for some things).#2018-10-1119:29Michael FianoPerhaps @U064X3EF3 can weigh in here as I see he is the project lead. When i get home later I'll open an issue if only to document this edge case so other newcomers don't bang their heads like I did.#2018-10-1119:33andy.fingerhutIt is better to ask about whether it can change when things are still alpha, than later. It is really up to folks like Alex, Rich, and Stu Halloway on how they choose to make the tool work, so don't be surprised with the answer "doc enhancement, please"#2018-10-1119:35seancorfieldMy sense is that it is "by design" that aliases are combined in order, left-to-right and that -A:foo is just shorthand for -R:foo -C:foo -M:foo ... -- so the behavior you observe is intentional and likely considered to be already well-documented.#2018-10-1119:36seancorfieldIf you specify your project-local alias with -R and/or -C instead of -A then you'll get the behavior you want.#2018-10-1119:37Michael FianoThe confusing bit is that -M is documented as only the last one being used, but if -A is indeed shorthand for that, an -A after an -M needs to have a note mentioning that only the last explicit -M is applied, and -A is ignored.#2018-10-1119:37seancorfieldThe -A is not ignored -- the aliases are all combined left-to-right, but with :main-opts it's always last one wins.#2018-10-1119:37Michael FianoBut that is not the case.#2018-10-1119:38Michael Fiano-M:foo -A:bar, when both foo and bar have :main-opts keys, then foo wins.#2018-10-1119:42seancorfieldAh, gotcha. Here's why: val="$(join '' ${resolve_aliases[@]})|$(join '' ${classpath_aliases[@]})|$(join '' ${all_aliases[@]})|$(join '' ${jvm_aliases[@]})|$(join '' ${main_aliases[@]})|$deps_data"
#2018-10-1119:43seancorfieldSo -M is always applied last. OK, yes, that should be documented. Do you want to create a JIRA ticket for that (against the CLJ JIRA project I believe)?#2018-10-1119:43seancorfieldI'm happy to create the ticket if you want me to.#2018-10-1119:44Michael FianoI very much want to so others don't run into the headaches I did. I won't be able to do that until tonight or tomorrow though when I'm at my dev machine, unless you don't mind.#2018-10-1119:46seancorfieldI'll go ahead and create it... against the website since it's really an error in the Reference page for the CLI#2018-10-1119:46Michael FianoThanks @seancorfield!#2018-10-1119:54seancorfieldhttps://github.com/clojure/clojure-site/issues/306#2018-10-1122:00alexmillerThis is a bug, not a doc issue. Order matters, should be respected.#2018-10-1123:33seancorfieldOK, good to know. So the shell script would need to expand each -A as it encountered it, adding the aliases as if all four were really specified explicitly?#2018-10-1123:33seancorfieldDo you want a JIRA issue (and maybe a patch) against CLJ for that instead?#2018-10-1123:46alexmillerJira yes, patch if you like. I suspect it affects the scripts too so may require some coordination#2018-10-1123:52seancorfieldUgh! Yes, fixing the clojure script would solve one side of things -- that's easy -- but tools.deps also needs to be fixed too. OK. JIRA for now. I'll look at a patch if I get enough free time but it's a bigger change than I had initially thought.#2018-10-1201:38seancorfieldhttps://dev.clojure.org/jira/browse/TDEPS-100#2018-10-1201:42Michael FianoThank you everyone.#2018-10-1201:27arrdemRe: yesterday’s discussion of depstar, this is apparently floating around https://github.com/luchiniatwork/cambada#2018-10-1201:37seancorfieldYeah, that is a lot more comprehensive but has a lot of knobs and dials -- and has its own set of surprises, I gather, from people who've used it.#2018-10-1206:03tianshuis that in tools.deps, there's no exclusions?#2018-10-1206:08seancorfield@doglooksgood :exclusions work fine in deps.edn#2018-10-1206:11seancorfieldDo you have a specific case where you think it doesn't work?#2018-10-1206:13dominicm@doglooksgood there isn't a top level exclusions though, which is something people try though, only a per-dependency exclusions.#2018-10-1206:15tianshu@seancorfield When using leiningen, usually I found there're some conflicts in my dependencies, but with clojure -Stree, I don't get any information like this.#2018-10-1206:15tianshuSo I don't know if there's conflicts or not#2018-10-1206:18dominicm@doglooksgood I think tools.deps operates on a different principle to maven/lein, so conflicts are less of an issue. tools.deps always picks the newest version, as opposed to maven which picks the first one it finds.#2018-10-1206:24tianshuSLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/tianshu/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j
/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/tianshu/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2
.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
I want to remove one of the binding, but don't know where introduce the multiple bindings.#2018-10-1206:27tianshuwith lein I got some information like this:
Possibly confusing dependencies found:
[org.quartz-scheduler/quartz "2.3.0"] -> [org.slf4j/slf4j-api "1.7.7"]
overrides
[conman "0.8.1"] -> [hikari-cp "2.4.0"] -> [com.zaxxer/HikariCP "3.1.0"] -> [org.slf4j/slf4j-api "1.7.25"]
and
[org.quartz-scheduler/quartz "2.3.0"] -> [com.zaxxer/HikariCP-java6 "2.3.13"] -> [org.slf4j/slf4j-api "1.7.10"]
and
[ch.qos.logback/logback-classic "1.2.3"] -> [org.slf4j/slf4j-api "1.7.25"]
Consider using these exclusions:
[conman "0.8.1" :exclusions [org.slf4j/slf4j-api]]
[org.quartz-scheduler/quartz "2.3.0" :exclusions [org.slf4j/slf4j-api]]
[ch.qos.logback/logback-classic "1.2.3" :exclusions [org.slf4j/slf4j-api]]
#2018-10-1206:28tianshuthen I know I can remove the binding by add exclusions on the org.quartz-scheduler/quartz dependency.#2018-10-1206:30dominicm@doglooksgood this is different to conflicting dependencies. clj -Stree gives you the information you need to fix this, you need to find which dependency is bringing in the logback you don't want, either nop of logback-classic.#2018-10-1212:25alexmillerI don’t think -Stree actually does provide info to do that - it just lists what is used, not what isn’t #2018-10-1212:25alexmillerThis is an area that needs some more work #2018-10-1212:26alexmillerIf you don’t like the one you’re getting, you can try excluding that one and see which one you get instead#2018-10-1214:57dominicm@alexmiller that's what I meant. It will show where logback-nop comes from.#2018-10-1216:02seancorfield@alexmiller Any thoughts on adding some sort of "pedantic" check to tools.deps?#2018-10-1218:04alexmillerin terms of reporting or also failing on pedanticness?#2018-10-1218:18Michael FianoAre there any plans to allow whitespace in strings?#2018-10-1218:19Michael FianoI was reading the edn spec and it says commas are not treated as whitespace within strings, but I must use them for arbitrary code using -e for example#2018-10-1218:36alexmillerwould like to allow, in practice it’s painful due to the interaction with the shell#2018-10-1218:37Michael FianoI see. Editor word-wise commands etc get confused so it's painful to edit#2018-10-1410:05isaacdoes deps.edn supports set env or properties?#2018-10-1410:41dominicm@isaac Yes. Although I'm not certain that I fully understand what you're asking.#2018-10-1410:43dominicme.g.
❯ clj -J-Dx=y -e '(System/getProperty "x")'
"y"
works.#2018-10-1410:48isaac:local/root "${LIB_ROOT}/xxx.jar"#2018-10-1410:48dominicm@isaac no, this doesn't work.#2018-10-1410:51isaacthere have difference normally between local dev env and production env#2018-10-1410:53isaacIMO, deps.edn supports env variable (or property) is necessary.#2018-10-1410:54dominicmYou can emulate this by using -Sdeps. e.g. clj -Sdeps "{:deps {foo/bar {:local/root \"${LIB_ROOT}/xxx.jar\"}}}"#2018-10-1410:54dominicmNormally we solve this by using different aliases for dev and production.#2018-10-1411:17isaacwhy :main-opts ["-e" "(println 'hello)"] throws EOF while reading?#2018-10-1411:24isaacit seems the :main-opts splits arguments by space#2018-10-1412:18mfikes^ Known issue https://dev.clojure.org/jira/browse/TDEPS-56#2018-10-1412:23mfikesWorkaround with Corfield comma:
:main-opts ["-e" "(println,'hello)"]
#2018-10-1412:50aisamu>Corfield comma
🙂#2018-10-1418:32richiardiandreaCan clojure conjoin vectors in deps.edn ? I have a feature request in nrepl/nrepl that could really use that: https://github.com/nrepl/nREPL/issues/55#2018-10-1419:25dominicmYour vector is just a string#2018-10-1419:27dominicmYou may be able to compose them, but I don't think so. #2018-10-1419:27dominicmI think the correct fix is multiple -m flags#2018-10-1420:25richiardiandreaYep i agree! See suggestion in the nRepl issue 😄 #2018-10-1510:10dominicmUnexpected side effect of gitlibs with "optional" namespaces, e.g. for myapp.ns.spec. tools.namespace will try and load the namespace regardless. This is also frustrating for :local/root dependencies, where I actually want to be able to refresh and work on both in parallel, but not necessarily on the specs.#2018-10-1510:27dominicmThis is similar to what clojure.core goes through I suppose.#2018-10-1515:31danielnealI'm doing something derp-y here - can anyone halp - what have I missed out?
clj -Sdeps '{cheshire {:mvn/version "5.8.1"}}'
Clojure 1.9.0
user=> (require '[cheshire.core :as json])
FileNotFoundException Could not locate cheshire/core__init.class or cheshire/core.clj on classpath. clojure.lang.RT.load (RT.java:463)#2018-10-1515:32alexmillerclj -Sdeps '{:deps {cheshire {:mvn/version "5.8.1"}}}'#2018-10-1515:32alexmiller^^ missed :deps layer#2018-10-1515:32danielnealah awesome! Thanks#2018-10-1515:32danielnealso excited using the cli for the first time#2018-10-1515:33alexmillerI’ve been thinking about adding a warning as this is the nth time someone has asked a question like this#2018-10-1515:33alexmiller(inc n)#2018-10-1515:33danielnealhaha so yup I can't read but at least it's not just me#2018-10-1515:33danielnealwarning++#2018-10-1515:34danielnealso nice just to get a quick clojure repl to prod this json object - I never can remember how to get jq to do what I want.#2018-10-1516:01seancorfieldYeah, I've been using clj for ages and I still do this, omitting the :deps layer! Probably at least once a day.#2018-10-1516:15mfikesAnother warning that could prove useful is if you omit the :mvn/version layer as in
{:deps {cheshire "5.8.1"}}
I do this, say, once per week.#2018-10-1516:15seancorfieldYou do get an error for that -- but it's a bit obscure.#2018-10-1516:16mfikesAhh, cool#2018-10-1516:16seancorfieldError building classpath. Coordinate type not loaded for library cheshire/cheshire in coordinate "5.8.1"
#2018-10-1516:17seancorfield(and, yeah, I do that all the time too!)#2018-10-1516:20orestisSounds like a need for some kind of tool that allows you to write specifications for data?#2018-10-1516:20mfikesYeah, one that is in alpha 🙂#2018-10-1516:22orestisI’ve never typoed the maven thing, but it could be I’m just editing an existing file with plenty of examples. #2018-10-1516:36andy.fingerhutor perhaps a lint-like tool that has custom non-spec code that looks for and warns about data structures that look like they may be mistakes...#2018-10-1516:40andy.fingerhutFYI, a transcript of Alex Miller's "Dependency Heaven" talk is now available here: https://github.com/matthiasn/talk-transcripts/blob/master/Miller_Alex/DependencyHeaven.md#2018-10-1516:51seancorfieldWe've made a lot of progress in this last week on moving from Boot to clj/`deps.edn` for our monorepo.#2018-10-1516:52seancorfieldThe issues with relative paths in :local/root has meant we have to have everything "side by side" in one part of the repo.#2018-10-1516:53seancorfieldAnd we found we really needed a baseline deps.edn (in the project) and a per-subproject deps.edn (for each subproject in the repo).#2018-10-1516:54seancorfieldSo we ended up having a dummy subproject and pointing CLJ_CONFIG at that folder, so we can have all our "pinned" versions in :override-deps in an alias that every command has to use CLJ_CONFIG=../versions clj -A:defaults:other:aliases
#2018-10-1516:55seancorfieldWe have a small "build" tools subproject with a bunch of -main entry points for various tasks and we bring it in with aliases (in ../versions/deps.edn).#2018-10-1516:58seancorfieldWe'll probably have a couple of shell scripts to provide some sugar over those invocations but we've got nREPL capabilities, test, build a JAR, run migrations (SQL/data) all up and running with "just" clj and a few aliases. I need to convert two more pieces this week and we'll be close to just removing Boot from our pipeline.#2018-10-1517:17lwhortonhey with the new tools.deps i’m tooling around with a local jar file via the :local/root "/path/to/jar", but changes to the jar don’t seem to be reflected in the repl session. what do i have to do to (reload) or re-require after changing the jar so that the latest code is around in the repl?#2018-10-1519:13alexmillerI’m not sure that you can get the default Java classloaders to do this - they don’t pick up modified jar files after startup. So even reloading your Clojure namespace is probably not going to help afaik.#2018-10-1519:30lwhortonthat seems to be what’s happening. for now we’re just hard-restarting the whole repl#2018-10-1520:15alexmillerSomething like a boot pod would give you a fresh classloader and the ability to do this#2018-10-1517:29seancorfieldThat sounds like a plain ol' REPL workflow issue and you'd need to tell Clojure to reload namespaces from that JAR, regardless of which tool you were using.#2018-10-1517:29seancorfieldMind you, if you're rebuilding the JAR locally, why not point :local/root at the source code instead and remove the JAR step?#2018-10-1517:30seancorfield(you'll still need to tell Clojure to reload the code -- after all, the classpath is not changing and Clojure doesn't know to re-read files on the classpath unless you tell it to)#2018-10-1517:33lwhorton> reload namespaces from that JAR
is that a good old (require '[foo.bar]) again?#2018-10-1517:56mfikes@lwhorton (require '[foo.bar] :reload)#2018-10-1518:02dominicm@seancorfield what layout did you have before? #2018-10-1518:13seancorfield@dominicm We had a top-level deps.edn (equivalent) and our misc. build tooling was in a completely separate folder build/
<tooling here>
clojure/
deps.edn equivalent
subproject-a/
deps.edn
subproject-b/
deps.edn
...#2018-10-1518:17seancorfieldNow we have build/
<wrapper shell script>
clojure/
build/
deps.edn
build/dev-related Clojure code
versions/
deps.edn -- pinned versions and aliases
subproject-a/
deps.edn
subproject-b/
deps.edn#2018-10-1518:19dominicm@seancorfield not super different then. Interesting solution using CLJ_CONFIG. Hard to get people to set that up, but not a problem in a smaller team though. #2018-10-1518:20seancorfieldThat's why we have a wrapper shell script 🙂#2018-10-1518:23seancorfieldThe shell script is basically build task subproject and it maps mostly onto (cd path/to/<subproject>; CLJ_CONFIG=../versions clojure -A:defaults:<task>)
#2018-10-1518:24seancorfieldbuild nrepl datamapper or build test environment or build migrate sql etc#2018-10-1518:43dominicm@seancorfield that doesn't seem very amenable to cursive or ciders jack in? How are you editing? #2018-10-1519:14seancorfieldWell, I'm using Atom/ProtoREPL which doesn't support clj in any useful way and my colleague who uses Emacs/CIDER always starts his REPLs by hand and then connects to it via CIDER...#2018-10-1519:15seancorfieldBecause of the whole CLJ_CONFIG=../versions clojure -A:defaults thing, it's a pain to configure tools to start clj-repls that way anyway.#2018-10-1519:33dominicmYeah. It is as a result of that 🙂. You could restore that functionality by automating some kind of alias merging across your deps.edn files.#2018-10-1519:33dominicm(This is something I expect I will end up writing, but I'm still fiddling with ways to avoid it)#2018-10-1519:47seancorfieldYou'd either need to have support for multiple deps.edn files in a clj command or your user deps.edn file would need everything we have in versions/deps.edn tho'... (if you mean to remove CLJ_CONFIG=../versions from the command itself)#2018-10-1519:47seancorfieldEither way, not a problem for the sort of usage we have/need.#2018-10-1519:47dominicmI understand. I was looking to you for a silver bullet, as I'm currently getting Edge more & more tools.depsy.#2018-10-1519:48seancorfieldThere are no silver bullets (tm)!#2018-10-1519:48dominicmI can feel Fred Brooks channeling through you#2018-10-1520:40andy.fingerhutWhat are your views on bronze bullets? 🙂#2018-10-1521:02arrdem@seancorfield I’d be curious what you think of https://github.com/arrdem/katamari - I just got it to the point of being able to build classpaths, uberjar and self-host last night. It’s designed to solve the sorts of many targets and shared configuration in a single monorepo friction you’re encountering as I hit them too in my first pass at deps for our monorepo.#2018-10-1521:34zane🎶 Na na~ 🎶#2018-10-1521:08seancorfieldWill take a look. Thanks @arrdem#2018-10-1521:19arrdemIt’s extremely pre-alpha and currently extremely under-documented, but I’m happy to answer questions and will be trying to clean it up as I go.#2018-10-1602:03kennyI have this code:
(ns where-print
(:require
[clojure.spec.alpha :as s]))
(s/def ::foo string?)
(defn -main
[& args]
(when-not (s/valid? ::foo 1)
(s/explain ::foo 1)
(System/exit 1)))
If I run clj -m where-print nothing is printed to stdout. Any idea how to prevent this from happening?#2018-10-1602:45seancorfield(System/exit 1) is a pretty brutal way to exit the program and it can quit the JVM before output is flushed. If you add (flush) before that exit, you should get your output.#2018-10-1602:49seancorfieldIt would also work if you did this: (defn -main
[& args]
(when-not (s/valid? ::foo 1)
(println (s/explain-str ::foo 1))
(System/exit 1))) since println will flush stdout to the newline (although your output will now have two newlines) @kenny#2018-10-1603:07kenny@seancorfield thanks. By brutal, are you hinting at a better way to exit the process with a unsuccessful exit code?#2018-10-1603:14seancorfieldNo. Just observing that it it immediately kicks off a shutdown of the JVM and some things may not get done that you are expecting 🙂#2018-10-1606:01dottedmagJVM — Unleash your inner Hulk!#2018-10-1707:37dominicmSimilar data point to knowing about conflicts, I'd really like to know if a dependency wants a newer version of a dependency than I am giving it.
project depends on b 0.1
project depends on a
a depends on b 0.2#2018-10-1718:55seancorfieldAs of this morning, our entire CI pipeline (including SQL migrations, JAR building and deployment to servers) runs on clj/`deps.edn` for most of our projects -- previously that was all on Boot.#2018-10-1809:03martinklepschdo you also build jars for libraries which you push to a maven repo? I find the existing libs mostly focus on uberjars or don't help with the creation of the pom file/actual deployment.#2018-10-1814:49seancorfieldWe only create uberjars for deployment. #2018-10-1720:10ghadidang!#2018-10-1720:20alexmiller!#2018-10-1720:20alexmillerNew tools.deps.alpha and clj out: https://groups.google.com/d/msg/clojure/xCa1mU2sOWs/XcEAGJ4RBQAJ#2018-10-1720:21alexmillerMostly a fix/recommendation around connecting to private s3 maven repos#2018-10-1811:55pauldHow can I debug the clj tool chain? clj hangs when I put org.openjfx/javafx-graphics {:mvn/version "11"} in the deps.edn file.#2018-10-1811:59gfredericksdid it successfully download the jar?#2018-10-1812:02bronsaI see a massive cpu spike after clj finishes downloading the artifact#2018-10-1812:04pauldI don't know. I can get it working it boot, so boot was able to download the jar.#2018-10-1812:04pauldmaybe I need to wait longer? Still it works just fine in boot, so there is some issue here with the clj tool.#2018-10-1812:09bronsait looks to me like there's some infinite loop during dependency resolution#2018-10-1812:10mpenetThe latest clj should fix that#2018-10-1812:10mpenetAt least there was an issue related to this#2018-10-1812:10bronsayeah I remember that, but I'm on latest tools.deps#2018-10-1812:10mpenetTDEPS-79#2018-10-1812:10mpenetoh #2018-10-1812:10bronsaat clojure.lang.APersistentVector.hasheq(APersistentVector.java:162)
at clojure.lang.Util.dohasheq(Util.java:177)
at clojure.lang.Util.hasheq(Util.java:168)
at clojure.lang.PersistentHashMap.hash(PersistentHashMap.java:120)
at clojure.lang.PersistentHashMap.valAt(PersistentHashMap.java:152)
at clojure.lang.PersistentHashMap.valAt(PersistentHashMap.java:156)
at clojure.lang.RT.get(RT.java:753)
at clojure.core$get.invokeStatic(core.clj:1492)
at clojure.core$get.invoke(core.clj:1486)
at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
at clojure.core$reduce1.invokeStatic(core.clj:934)
at clojure.core$get_in.invokeStatic(core.clj:6062)
at clojure.core$get_in.invoke(core.clj:6062)
at clojure.tools.deps.alpha$excluded_QMARK_.invokeStatic(alpha.clj:87)
at clojure.tools.deps.alpha$excluded_QMARK_.invoke(alpha.clj:83)
at clojure.tools.deps.alpha$include_coord_QMARK_.invokeStatic(alpha.clj:113)
at clojure.tools.deps.alpha$include_coord_QMARK_.invoke(alpha.clj:106)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:180)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:165)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:228)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:210)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
#2018-10-1812:11bronsahere's a thread dump as clj hangs#2018-10-1812:12bronsaoh that ticket hasn't been resolved yet @mpenet#2018-10-1812:12bronsaso maybe it's exactly that#2018-10-1812:19mpenetPatched was applied tho i think#2018-10-1812:20pauldI'm using .397#2018-10-1812:20bronsait's been reverted @mpenet#2018-10-1812:20bronsahttps://github.com/clojure/tools.deps.alpha/commit/b03942bbc75780537d8bfd91535b3a21d79a93ee#2018-10-1812:21mpenetAh#2018-10-1812:25alexmillerIt was accidentally applied - I am working on alternate patch#2018-10-1812:30pauldMore background: https://dev.clojure.org/jira/browse/TDEPS-79#2018-10-1813:07alexmiller@pauld can you drop a comment on TDEPS-79 with your deps.edn so I can double-check it is the same thing?#2018-10-1813:09pauldsure#2018-10-1813:19pauld@alexmiller done#2018-10-1813:20alexmillerthx#2018-10-1816:12hlollipart java question, part tools.deps question. I'm trying to leverage the git/url dep functionality on this java repo
com.kunstmusik/CsoundJNA {:git/url ""
:sha "a717dc3af6bf268b2962cac854d3fa9c0e2901a9"}
it's downloaded to ~/.gitlibs and is added onto the classpath, all good there.
But the importing is the tricky part, knowing little about java, could it be that the pom.xml file is ignored or is missing some information, https://github.com/kunstmusik/CsoundJNA/blob/master/pom.xml or is my import on the java class wrong?
(:import [com.kunstmusik.csoundjna CsoundJNA])
in my deps.edn file I have this as well
org.clojure/tools.deps.alpha {:mvn/version "0.5.460"}
#2018-10-1816:18alexmilleryou can’t use a Java dep like this as something has to compile the java file to a class file. For that, you either need a build tool or an artifact with compiled classes in it.#2018-10-1816:18alexmillergit deps are only going to be useful with Clojure source only (which does not require a build)#2018-10-1816:19hlolliah, I understand now. Thanks @alexmiller! I'll ask my friend to just publish it to maven, probably easier 🙂#2018-10-1818:01bastiIs there a way to check for the latest version of dependencies defined in deps.edn?#2018-10-1818:01dominicmTool named depot by olical @basti #2018-10-1818:08bastiThanks I’ll check it out!#2018-10-1819:32seancorfieldWhy does (reader/clojure-env) care about actually building a classpath?#2018-10-1819:35seancorfieldBackground: we use an env var for CLJ_CONFIG so we have a known "baseline" for aliases and override-deps etc; but our actual deps require an alias -- -Adefaults -- to resolve since they all have {} for the coords and the actual versions are provided by the alias and :override-deps. So we can run CLJ_CONFIG=../versions clojure -Adefaults -Sdescribe
but if we omit the alias, (reader/clojure-env) blows up with Error building classpath. Coordinate type not loaded for library selmer/selmer in coordinate {}
Because of course we can't pass the alias into the clojure-env shell call to clojure -Sdescribe#2018-10-1819:36seancorfield@alexmiller Is there a reason that describe needs to build the classpath? If so, would you be open to a patch to allow additional arguments to be passed into clojure-env to support required aliases etc?#2018-10-1819:39dominicmI think I was supposed to open a ticket for that, sorry#2018-10-1819:43seancorfieldLooks like it fails because it thinks it is "stale" and therefore runs the classpath generation code.#2018-10-1819:55alexmiller@seancorfield there is a ticket for this already (and no, I don’t think so)#2018-10-1819:56alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-82#2018-10-1819:58seancorfieldYup, confirmed. I modified my local clojure script to set stale=true when describe is true, and also set cp= to avoid loading the (possibly non-existent) .cpcache file, and then clojure-env runs as expected in my environment.#2018-10-1821:21seancorfieldI attached a patch to that ticket matching what I changed locally to make that work @alexmiller /cc @U09LZR36F#2018-10-1821:21alexmillerthx!#2018-10-1916:46seancorfield@alexmiller Do you think TDEPS-82 will get fixed and released "soon"? We have an "outdated dependencies" step in our build chain that we'd like to run other places that just my dev machine so we'd need an updated clojure release for that.#2018-10-1918:13alexmillerI would like to fix TDEPS-79 and would then be happy to pull 82 along with that into a release#2018-10-1819:55ro6@alexmiller I think the documentation formatting is broken at https://clojure.org/reference/deps_and_cli#_maven_s3_repos#2018-10-1819:56alexmillerthx, I just modified that yesterday, will fix#2018-10-1819:58ro6Looks like maybe the code block was supposed to end after <servers>
...
<server>
<id>my-private-repo</id>
<username>AWS_ACCESS_KEY_HERE</username>
<password>AWS_SECRET_ACCESS_KEY_HERE</password>
</server>
...
</servers>
but didn't.#2018-10-1819:59alexmilleryeah, just fixed#2018-10-1914:31jeff.terrellIs there a clj/clojure CLI equivalent to lein deps :tree?#2018-10-1914:55seancorfieldThe closest is clj -Stree but it doesn't show conflicts, just the resolved dependencies. Folks have asked for this a few times. You may be able to get more information with -Sverbose -Sforce. #2018-10-1914:56jeff.terrellThanks. Nice to know for sure that this feature doesn't exist (yet). And -Sverbose -Sforce doesn't illuminate the conflict.#2018-10-1914:58seancorfieldI'm not at my computer right now but there is a way to get tools.deps to tell you what versions it finds and what it selects -- which is close. #2018-10-1914:34jeff.terrellOr more generally: how can I diagnose a dependency conflict in a deps.edn project? I searched in various places (web, FAQ, deps & CLI guide and reference) but didn't see an answer. Seems like the kind of relatively undocumented thing that might go in a FAQ entry, which I'm willing to write if there's a good answer.#2018-10-1914:38jeff.terrellAnd by the way, is there a canonical adjective to use for projects that manage dependencies with deps.edn? I'm looking for something parallel to 'A Leiningen project' and 'A Boot project'. A 'tools.dep' project? A 'deps.edn' project? A 'clojure deps' project?#2018-10-1914:57seancorfieldI've been calling them 'A clj project' to parallel the use of the tool to run them but there's nothing official as far as I know. #2018-10-1914:58jeff.terrellThat's better than circumlocution, and maybe not too ambiguous. simple_smile Thanks.#2018-10-1915:07alexmillerI would say not tools.deps, but either deps.edn or Clojure tools project#2018-10-1923:55jimmyIs there a way to run clj and just have it pull down dependencies listed in deps.edn? Not start a repl or run anything, just download the dependencies.#2018-10-2000:00lilactownI run clj -Stree to do that#2018-10-2000:29alexmiller-Spath#2018-10-2005:36richiardiandreaSilly question: when clojure emits a Downloading: I guess it did not find the dep already in .m2 right?
I am trying to figure out why I have some many "Downloading" in my CI build#2018-10-2005:42richiardiandreaYeah checking the code it seems so. It seems to me very odd that I am downloading the same deps over and over. There must be some maven setting wrong#2018-10-2005:43richiardiandreaRunning on Azure Pipelines I wonder if somebody has had any problem#2018-10-2012:21alexmillerIn CI, often you don’t get the .m2 dir preserved. Some providers let you mark that in different ways.#2018-10-2015:23richiardiandreaIt is odd because I am running it all the tests/commands in the same shell#2018-10-2218:39jeff.terrellIf you have a deps.edn project, how do you push it to Clojars?
The Clojars "pushing" wiki page, for example, gives instructions for Leiningen, Boot, and Maven, but not for deps.edn:
https://github.com/clojars/clojars-web/wiki/Pushing#2018-10-2218:40seancorfieldYou don't 🙂#2018-10-2218:40seancorfieldA deps.edn project can be depended on directly on GitHub.#2018-10-2218:41seancorfieldA more helpful answer to your question is that you will need to use something like cambada to create a "thin" JAR and then do something appropriate to generate pom.xml and deploy it up to Clojars (Maven, I guess?).#2018-10-2218:59jeff.terrellThanks, Sean. Yeah, I knew that about depending directly on GitHub commits. simple_smile But since the CLI tools don't work on Windows yet, I'm using Leiningen on my project, and Leiningen can't depend on GitHub coordinates (even with the lein-tools-deps plugin).#2018-10-2219:00jeff.terrell(I don't use Windows, but some members of my team do, and installing a Linux environment is a bit of a tall ask at this point.)#2018-10-2219:09seancorfieldI'm surprised lein-tools-deps can't depend on git coords... boot-tools-deps supports those...#2018-10-2219:13jeff.terrellIt can depend on git coords, but that functionality doesn't work on Windows. It complains about not being able to find the clojure command.#2018-10-2219:24alexmilleryou can use clj -Spom to create a pom, which could be modified into a deployment path too#2018-10-2219:25alexmiller(in any answer to prior questions)#2018-10-2219:25alexmiller-Spom syncs, so if you change it the pom, those changes (outside the dependencies) will be respected#2018-10-2219:26jeff.terrellNice, good to know, thanks Alex.#2018-10-2317:37dominicmI might have a try tonight at getting pack to work with clojars (ish). The parts are there. #2018-10-2318:15dominicmHow desperate are people to avoid mvn the cli tool? I think pack already works with mvn#2018-10-2318:37seancorfieldmvn just has such a nasty CLI compared to, say, lein deploy or boot deploy...#2018-10-2318:41dominicmhttps://clojars.org/io.dominic/super-duper-octo-barnacle but it also works 😄 (almost first time)#2018-10-2318:42dominicmI find it a little weird that deploy-file doesn't look at pom.xml to figure out the url automatically, but meh#2018-10-2318:44dominicmI'll write up the process with mvn and put it in pack's readme, and point at the working example.#2018-10-2318:57dominicmhttps://github.com/juxt/pack.alpha#uploading-to-clojars-or-maven really simple 🙂 If someone knows a mvn-less way to upload to maven/clojars, I'm all ears. Seems like a good thing for a small targeted library to do really well (like with really secure credentials settings from day #1).#2018-10-2319:20seancorfieldTo save me the trouble of actually trying to build a skinny JAR with pack and opening it up to look inside, does it add a manifest and, if so, what attributes are in it?#2018-10-2319:27dominicmThe mvn deploy:deploy-file can take a pom.xml externally to use. I guess it uploads separately. You could put it inside the jar by adding a mkdir extra && cp pom.xml extra/ then passing -e extra to pack when building.#2018-10-2319:27dominicmI don't think things generally depend on the pom.xml being inside the jar though.#2018-10-2319:37seancorfieldSo... no manifest added nor needed?#2018-10-2319:38seancorfieldAnd what is the difference between a "regular" skinny jar from pack and one that also has --no-libs?#2018-10-2319:38dominicm@U04V70XH6 not inside the jar, nope. You do need a pom.xml, but it's generated out of band. I didn't actually check if it's in my jar, one sec, I will#2018-10-2319:38dominicmNo pom.xml in my jar. It runs fine.#2018-10-2319:38seancorfieldI'm thinking back to the discussion/complaints about depstar not adding even a basic manifest into the uberjar (so you couldn't just java -jar it).#2018-10-2319:39dominicm@U04V70XH6 That's a MANIFEST.MF, you can use the -e flag in pack to add it to your classpath at build time 🙂#2018-10-2319:39seancorfieldIt wasn't a big deal to add a minimal manifest with Main-Class but of course you don't actually need to -- you can just do java -cp ... clojure.main ... and run the JAR anyway.#2018-10-2319:40dominicm@U04V70XH6 regarding --no-libs, the general idea of skinny jar mode is to produce something that's really cheap to rsync or something onto s3. So you generally output:
app.jar
lib/clojure-1.8.0.jar
lib/github-0.1.0.jar
And then upload app.jar + any files from libs that are missing.
This was requested by a company doing java, this is their upload process.#2018-10-2319:40seancorfieldYeah, my point/question was "Does pack add a manifest by default?" and the answer seems to be "no" and, implied, does a library published to Clojars need a manifest and I think the answer is also "no".#2018-10-2319:41dominicmCapsule/Onejar do add a manifest, because it's required for them to function. I'm not entirely happy with the process though. I think more could be done here, but there's nothing to drive change currently.#2018-10-2319:41seancorfieldAh, so "skinny" means no libs in the JAR but instead copy them to a lib/ folder?#2018-10-2319:42dominicm@U04V70XH6 Yep!#2018-10-2319:43dominicmIn the future I might add a dedicated maven namespace to pack. I want to end up where you go via use-case rather than "underlying details". UberJAR complicates this a little, as there's choice.#2018-10-2319:44seancorfieldPart of me is just contemplating what I might do with my fork of depstar -- to keep it as minimal as possible but open up the possibility of producing a thin JAR and being part of a workflow that could produce a pom.xml and mvn it up to Clojars... or another repo...#2018-10-2320:23dottedmagPardon my ignorance, does Lein use mvn to upload?#2018-10-2320:24andy.fingerhutProbably more likely to find someone who knows the answer to that in #leiningen channel#2018-10-2320:26dottedmagFigured it out already 🙂 It uses cemerick/pomegranate - a wrapper around Eclipse Aether.#2018-10-2320:43arrdemtools.release(.alpha) 😉#2018-10-2403:18seancorfieldI gave in and put seancorfield/clj-new and seancorfield/depstar on Clojars. Releases 0.5.4 and 0.1.2 respectively. JARs built with (my fork of) depstar. Initial pom.xml built by clj -Spom. Pushed to Clojars with mvn deploy:deploy-file (ugh!).#2018-10-2417:18dominicmTopic of today. I want multiple aliases which compose and start multiple things. The goal would be using able to start a program like normal, but perhaps side load nrepl in.
Best idea I have so far is a custom main and jvm options. The custom main would evaluate code based on the provided options, and then finally run the -m it was provided.
Any other suggestions for ways to interface with such a thing?
I did consider taking a list of mains, but then you have to detect when they are done in an arbitrary way, but that might be fine by looking at the thread state? Also main-opts don't concatenate. #2018-10-2509:06codonovanYes. I think we had to use lein-figwheel "0.5.16" and figwheel-sidecar "0.5.14" to get it working. https://github.com/drapanjanas/re-natal/issues/190 might be helpful#2018-10-2515:20xfyreInteresting, let me check this in Xcode. I was experiencing reload issues before I even tried to upgrade Figwheel#2018-10-2515:34xfyreI’m seeing
2018-10-25 18:34:41.958455+0300 DcnetApp[90232:23128804] [] nw_socket_handle_socket_event [C224.1:1] Socket SO_ERROR [61: Connection refused]
#2018-10-2418:55seancorfield@dominicm I thought about this long and hard as we switched our dev/test/build pipeline over to clj from boot -- and ended up with the "custom main and jvm options" approach, with several small "microprojects" containing the various -main we want that are added via aliases... :main-opts don't concatenate for a (good) reason but that does sometimes make it hard to "compose" multiple tasks in a "build pipeline". Our compromise there was to create a build shell script that runs clojure for each set of command line arguments (loosely... it implements a sort of mini-DSL around aliases and subprojects to make it easy for us to kick off a sequence of tasks from a single command line).#2018-10-2418:56seancorfield(as of yesterday, we no longer use boot for any of our dev/test/qa/prod stuff -- with two legacy exceptions, one of which I'm working on right now 🙂 )#2018-10-2419:28dominicm@seancorfield Just so I am clear, you have a custom main which calls other -main? Or do you look at JVM opts in order to determine what to start in your one custom main?#2018-10-2419:29seancorfield"It Depends". We have some -main functions that use the supplied command line arguments to decide what to run (namespace-qualified function names, usually). But also, now we have our little build wrapper shell script, we often just run clojure multiple times with different (main) options...#2018-10-2419:30seancorfieldBut the former approach could easily use JVM options instead of command line arguments. Or as well as.#2018-10-2419:31seancorfield(the main issue is determining what arguments -- command line or otherwise -- to call each of the functions with, based on just a single -main entry point and its own command line arguments)#2018-10-2419:32arrdemit’d be interesting to have a “multi clojure.main” that took say an EDN array of mains and ran each one in its own future / send-off.#2018-10-2419:32arrdemor let you have “mixins” that get sent off and one “main”.#2018-10-2419:33dominicm@arrdem I'm toying with that idea. The hard limitation being that if you actually want the main to run once and then block, it's more difficult. Also, if you want ordering (e.g. producing some state picked up for the next main), then you have to "detect" when the main is done.#2018-10-2419:33arrdemConcatenating programs into a single JVM seems like … a job for a real program not some crazy main. We have a thing which does that, it’s a REPL 😉#2018-10-2419:34arrdemFor Katamari I’ve been toying with the idea of how to let kat reuse a JVM between taks if your dependencies haven’t changed. Not sure you can actually blow away enough Clojure state (in an unmodified Clojure) to do that safely or if it buys you anything.#2018-10-2419:35dominicmI'm trying to leverage existing standards (e.g. -main). But I agree. I'm feeling like I might have to bite the bullet and "force" something new onto people for daemons like nrepl.#2018-10-2419:36arrdemYeah. Remote daemonized nREPL servers with a client hooked to your terminal is the best I’ve got for stuff like this.#2018-10-2419:36arrdemI need to take a poke at pack.alpha, looks like it may provide more of the machinery I want for Katamari’s jar, uberjar and maybe pom targets than depstar does.#2018-10-2419:38dominicm@arrdem I'm trying to move towards having some kind of API for consumers. But I've prioritized -main support for now. I'm very welcome to improvements to the "api" 🙂#2018-10-2419:38arrdem@dominicm cool - I’ll see what I find 😉#2018-10-2419:38arrdemkat is already a pile of alpha APIs what’s one more.#2018-10-2419:39dominicmHehe.#2018-10-2419:39arrdem@seancorfield curious what wound up in your build script.#2018-10-2419:44seancorfield@arrdem Well, our basic command to run anything is essentially cd <repo>/clojure/<subproject>
CLJ_CONFIG=../versions clojure -A:defaults:<alias> <some> <args>
so mostly the build script maps pairs of <task> <subproject> on the command line into runs of that command pair.#2018-10-2419:45seancorfieldSince arguments are inherently variadic, we treat <task> <subproject> as a command with no args and [ <task> <subproject> <some> <args> ] as a command with some args. So build parses the [ and ] delimiters around commands.#2018-10-2419:46arrdemInteresting.#2018-10-2419:47seancorfieldPlus a couple of convenient conventions: <task>+test means "map <task> to its alias(es) and also add :test:test-deps; and <task>s means "run <task> for each of the (subproject) arguments provided" -- so you can easily build multiple uberjars or run multiple subprojects' tests.#2018-10-2419:48seancorfieldOh, and <task>-all which expands a list of subprojects to include all the others (recursively) that are pulled in as :local/root deps (so we can easily "test a subproject and everything it depends on")#2018-10-2419:49seancorfieldVery minimal really. And of course we could do it all from the command line but... we're lazy programmers...#2018-10-2419:50seancorfieldEverything beyond that "DSL" is handled with aliases (and command line arguments passed to clojure).#2018-10-2419:52arrdemHum. Do you just have standardized eg /test paths for the -all alias or how did you get dependees tests recursively included.#2018-10-2419:54seancorfieldA monorepo with subprojects all side-by-side in the clojure directory, each with src and test folders beneath them. Is that what you meant?#2018-10-2419:54arrdemYep.#2018-10-2419:54arrdemCool, thanks for the info!#2018-10-2419:54arrdemInteresting to see what other folks are coming up with.#2018-10-2419:55seancorfieldWe deleted most of our 2,000 line build.boot file yesterday 🙂#2018-10-2419:55seancorfieldI expect to delete almost all of the rest of it by week's end.#2018-10-2420:14dottedmag@seancorfield How big is the replacement?#2018-10-2420:16seancorfieldA very well commented and spaced out bash script with about 70 lines of help output -- total just under 300 lines 🙂#2018-10-2420:17dottedmagSo, 2000 lines of build.boot was just a dispatcher?#2018-10-2420:17seancorfieldNo... it's kinda hard to explain 🙂#2018-10-2420:18seancorfieldWe used to dynamically determine cross-subproject dependencies using tools.namespace -- seeing which namespaces a subproject required and which subprojects those were actually defined in. All that is gone and replaced with explicit :local/root dependencies now.#2018-10-2420:20seancorfieldPlus we had accrued a lot of now-unused build tooling as we evolved the project over several years (we switched to Boot at the end of 2015). We had a lot of per-subproject-specific stuff in the Boot file which was never made generic (which the shell script does).#2018-10-2420:20seancorfieldAnd, lately, this Boot bug was killing us https://github.com/boot-clj/boot/issues/715#2018-10-2420:21dottedmagI see.#2018-10-2420:23seancorfieldBoot had encouraged us to build complex tooling because it was easy to do so. The reason we switched from Leiningen to Boot three years ago was because we wanted to do more programmatic stuff -- which was much, much easier in Boot -- but we sort of got carried away 🙂 And moving to clj and deps.edn and aliases kind of focuses the mind on what's actually required.#2018-10-2420:31dottedmagIt reminds me of one of build steps (implemented in Go, to boot) I'm guilty of writing: it extracts from a Go binary a list of versions of packages it was built from, parses list of packages current module is dependent upon, and then decides whether that binary needs to be rebuilt. And to add to the insult, the list of binaries to check is created by parsing a Go source file.#2018-10-2420:33dottedmagVery nifty, cuts down maybe 10s from the full build, but it's more than 300 lines of code just for that.#2018-10-2420:34seancorfieldAh, yes, an easy trap to fall into 🙂#2018-10-2420:36arrdemCurious what makes you see that as a trap.#2018-10-2420:37seancorfieldWhen your build is "just code" it's easy to just start writing code -- without thinking about whether it is a) needed or b) an over-complicated way to achieve your goal.#2018-10-2420:51dottedmagAlso this code is usually easy and fun to write, so one can waste enormous amount of time on it.#2018-10-2505:09steveb8nI’m trying to use a private bitbucket repo as a git dep but getting “auth fail” from the jgit layer. I’ve tested ssh access from the terminal and that’s working. anyone else had any luck with this?#2018-10-2505:11seancorfieldI think that is only partially supported at he moment...?#2018-10-2505:11seancorfield(I'm trying to find the public discussions about it)#2018-10-2505:12seancorfieldHmm, OK, looks like ssh keypair should work for git deps...#2018-10-2505:13seancorfieldSee Procurers > Git Configuration here https://www.clojure.org/reference/deps_and_cli#_procurers#2018-10-2505:13seancorfieldIn particular "Note: user/password authentication is not supported for any protocol."#2018-10-2505:14seancorfieldHow do you have your repo configured in deps.edn @steveb8n?#2018-10-2505:15steveb8nthx for quick response. I’m using an SSH key with ssh-agent on OSX i.e. not using un/pw#2018-10-2505:15steveb8nin deps.edn I follow the pattern I’m using for some public repos i.e. same keys#2018-10-2505:15steveb8nonly difference is :git/url is ssh: instead of https:#2018-10-2505:16steveb8nwhen I run ssh -v in CLI, that shows successful connection#2018-10-2505:19steveb8nhmm, it looks like I am missing a “registered identity”#2018-10-2505:19steveb8nI’ll dig deeper into that#2018-10-2505:19steveb8nand will report back#2018-10-2505:30steveb8nI found the problem. I am using the OSX ssh-agent with the OSX keychain and it’s not persistent between reboots#2018-10-2505:30steveb8nssh-add -K fixed it#2018-10-2505:30steveb8nthanks for pointing me in the right direction#2018-10-2505:33seancorfieldCool! The whole ssh keys/identity thing can be a bear...#2018-10-2505:44steveb8nyep. in this case it’s a different behaviour in zsh CLI vs jgit (part of tools.deps). I’ve written it down in my readme for next time 🙂#2018-10-2510:47steveb8n@seancorfield a heads up for future. when using git private deps in CI, you must invoke the ssh-agent before invoking clojure i.e. eval clojure -A:dev only gives me this:#2018-10-2508:14Christian Johansenclj -A:dev
2018-10-25 10:06:47.433:INFO::main: Logging initialized @10380ms
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
#2018-10-2508:14Christian Johansencan this be improved somehow?#2018-10-2508:14Christian Johansena stack trace, anything to point me towards the problem?#2018-10-2508:19dominicm@christian767 my immediate thought is to try Clojure 1.10 alphas. As there's been some big changes around this stuff. #2018-10-2508:20Christian Johansenthis is with 1.10.0-alpha8#2018-10-2508:20Christian JohansenI’ll try an even newer one#2018-10-2508:30Christian Johansenwith clojure-tools-1.9.0.397 and Clojure 1.10.0-beta4:#2018-10-2508:31Christian Johansenclj -A:dev
2018-10-25 10:30:49.453:INFO::main: Logging initialized @11217ms
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
#2018-10-2508:31Christian Johansen😢#2018-10-2508:32seancorfieldSo that's failing before it even gets Clojure up and running...#2018-10-2508:32seancorfieldYou'll need to share some of your deps.edn (possibly both your ~/.clojure version and your project one).#2018-10-2508:33seancorfieldDoes it start a REPL without -A:dev?#2018-10-2508:33seancorfieldIf so, your :dev alias is the problem.#2018-10-2508:33Christian Johansengood tip#2018-10-2508:33Christian Johansensure does#2018-10-2508:33seancorfieldOK, so what's in your :dev alias?#2018-10-2508:33Christian Johansenfound the culprit#2018-10-2508:34Christian JohansenI have dev/user.clj, and in it I’m still using a dependency I ditched yesterday#2018-10-2508:34Christian Johansen[yonatane.timbre-json :as timbre-json]#2018-10-2508:35Christian Johansenis there something I could’ve done to have this reported in an understandable way?#2018-10-2508:35Christian Johansen(there was an :extra-paths in :dev)#2018-10-2508:35seancorfieldMaybe wrap the contents of your user.clj file in (try ... (catch Throwable t (println "dev/user.clj broke" t)))#2018-10-2508:36Christian Johansencan you try around the ns form?#2018-10-2508:36Christian Johansenanyway, what if the error was in my source? can’t try-catch all my code#2018-10-2508:36seancorfieldSure. ns is just code.#2018-10-2508:37Christian Johansenseems strange to have to try-catch all my code to see compilation failures#2018-10-2508:37seancorfieldWell, true, if your user.clj won't actually compile that's just going to break. user.clj is pretty special so you really should be very sparing about what you do in it.#2018-10-2508:37seancorfieldYou could always debug it with clj dev/user.clj I think?#2018-10-2508:38Christian Johanseninteresting#2018-10-2508:38seancorfield(without the -A:dev alias, it wouldn't be on your classpath so it wouldn't automatically run at Clojure startup)#2018-10-2508:38Christian Johansenmy impression was that I always got this behavior with any compilation errors, but you’re saying this is happening because it’s in user?#2018-10-2508:39seancorfieldRight, because it's executed as part of starting Clojure.#2018-10-2508:39Christian JohansenI see#2018-10-2508:39Christian Johansenby extension, this would happen with anything required from user too then?#2018-10-2508:40seancorfieldWell, wrapping the ns form in try/`catch` should deal with that.#2018-10-2508:40Christian Johansenright#2018-10-2508:40Christian Johansenleiningen manages to report this more clearly, I wonder why that is#2018-10-2508:40seancorfieldBecause Leiningen starts two JVMs perhaps?#2018-10-2508:41Christian Johansenaha#2018-10-2508:41seancorfieldOK, it's crazy late hear. I'm off to bed (nearly 2am).#2018-10-2508:41Christian Johansenwhoa, It’s 10 in the morning over here 🙂#2018-10-2508:41Christian Johansenthanks for your help#2018-10-2508:41seancorfieldHopefully other folks will be coming online soon if you have other Qs.#2018-10-2508:42Christian Johansen:+1:#2018-10-2509:16Christian JohansenI reported this as an enhancement: https://dev.clojure.org/jira/browse/TDEPS-102#2018-10-2509:16Christian Johansenit’s quite the buzzkill for new tools.deps users#2018-10-2510:40Christian Johansennot putting code in user.clj seems like the better approach for now. is there a tools.deps analog to leiningens :repl-options {:init-ns my-app.dev}?#2018-10-2510:43dominicmIn edge we solve this by having a dev.clj#2018-10-2510:50Christian Johansenhow do you load it?#2018-10-2512:24alexmillerI suspect this is really a Clojure problem, not a tools.deps problem#2018-10-2512:26Christian Johansenyes, after discussing with more people I realize it probably is#2018-10-2512:26Christian Johansensorry for misplacing the Jira issue…#2018-10-2512:46alexmillerNo worries, it can be moved#2018-10-2519:29arrdemthanks for the depstar fix @seancorfield - it works like a charm!#2018-10-2519:32seancorfieldI plan to actively maintain my fork, going forward, because we're relying heavily on it at work.#2018-10-2519:33seancorfieldI've also started using it for packaging library JARs for deployment to Clojars now (there's an hf.depstar.jar entry point for thin JARs now, then clj -Spom -- and one-off updates to add/update license, SCM, etc -- and then mvn deploy:deploy-file).#2018-10-2520:33arrdemCool. I’ll probably be looking long and hard at depstar vs pack.alpha shortly just because I need to start generating manifests from Katamari, but if I stick with depstar I’d be happy to support your copy if its intent is to be active.#2018-10-2520:46ghadiMy experiment worked!#2018-10-2520:47ghadiThe experiment was: Git deps allow you to be a shitty maintainer because someone can just fork and change the url, without having to deploy an artifact#2018-10-2520:48ghadiI learned this by making a simple PR to cognitect/test-runner that was ignored for months:
https://github.com/cognitect-labs/test-runner/pull/16
but I had no problem because I forked it internally and pointed to our un-busted fork#2018-10-2522:52zaneI've been waiting for that PR to be merged. 😞#2018-10-2520:49ghadi@seancorfield didn't have to do anything to unblock himself on depstar, which I neglected to maintain#2018-10-2520:50arrdem🤔 maybe {:mvn/version } could let you pass it an override lib to get the same place without refactoring your deps lists …#2018-10-2605:30dominicmThere's a ticket for this#2018-10-2521:03seancorfield:rolling_on_the_floor_laughing: @ghadi Although I have now deployed artifacts of seancorfield/depstar -- using the modified version of depstar! 🙂#2018-10-2521:04ghadiif you break the API in a significant way I'll kindly ask you to rename#2018-10-2521:04ghadibut I'm happy that you're happy sean!#2018-10-2521:05ghadiIt's a really good name if I do say so -- it's tar for deps, and a deathstar#2018-10-2521:07seancorfieldI like the simplicity of it. So far, I've only fixed bugs in the existing code and added a new namespace/entry point (for thin JARs). No API changes 🙂#2018-10-2522:52steveb8nwhen I use a :local/root dep, I’m finding that all my aliases are unavailable. workaround is to add copies of all aliases (with empty maps) into the dependencies deps.edn. Is this known?#2018-10-2523:22arrdem@steveb8n AFAIK this is intended behavior - clj/`clojure` do not yet support merging “shared” extra deps files eg. for aliases besides the install global file and the user config.#2018-10-2523:23arrdemI think seancorfield has done some hacking to be able to share a defaults file within a monorepo as have I.#2018-10-2523:25steveb8nhmm, in that case 1/ it would be good to doc this in the :local/root docs and 2/ does the workaround I discovered make sense?#2018-10-2523:26steveb8nI’m good to go with the workaround but just asking for others who will inevitably hit this#2018-10-2523:30steveb8nthat’s my version of “asking for a friend”#2018-10-2523:42seancorfieldIt doesn't make sense for aliases in a deps project you depend on to bleed into your project's deps.#2018-10-2523:43seancorfield:local/root is nothing special -- just a want to specify a dependency. You wouldn't want all the aliases from a random :git/url deps file to bleed into your project, would you?#2018-10-2523:44seancorfieldThe aliases available to you should always be: system deps + user deps (which can be anywhere based on CLJ_CONFIG env var) + project deps.#2018-10-2523:58arrdemI’ve been kicking around whether it makes sense to want to build deps with aliases - eg what if I wanted to let every one of my dependencies apply its :test alias so that I get a single classpath with all the machinery required to run all my deps’ tests.#2018-10-2600:00arrdemKinda gives you a way to have a parameterized debug build or such.#2018-10-2600:04steveb8ninteresting idea. I guess we’ll see some best practices emerge around primary and dependent aliases#2018-10-2607:09olivergeorgeThis is probably a silly question. Should it be possible to clone clojure locally and use it via deps.edn via :local/root?#2018-10-2607:24dominicmNope! It has Java in#2018-10-2607:26olivergeorgeThanks. Not really surprised. I did get it working by building first (e.g. ant install) and pointing at the jar with {:deps {org.clojure/clojure {:local/root "/Users/olivergeorge/repos/fork-clj/clojure/target/clojure-1.10.0-master-SNAPSHOT.jar"}}}#2018-10-2607:26olivergeorgeWas wondering more broadly how easy it was getting to tweak clojure source and see it in a REPL.#2018-10-2612:40alexmillerI do something similar when working on Clojure#2018-10-2621:49olivergeorgeCool. Pretty productive given the clojure build process is all of 10s on a 4yo macbook#2018-10-2706:24dominicmhttps://maven.apache.org/shared/maven-artifact-transfer/apidocs/index.html if someone wanted to write a tools.release, the maven library exists to do that 🙂#2018-10-2706:44dominicmI've got a complete first pass of "ednup" a tool for rewriting edn without losing whitespace. It's not great, but it is definitely sufficient to handle resolve tags! Don't expect this to work beyond this commit, I'm going to clean up with new namespaces, but:
{:paths ["src"]
;; cool comment
:deps
{potemkin {:mvn/version "0.1"}
tools.deps.alpha {:git/url ""
;; totally rad
:tag "tools.deps.alpha-0.5.460"}
;; woah dude
tools.deps.alphax {:git/url ""
:tag "tools.deps.alpha-0.5.460"
;; zoom
:sha nil}}}
Running
~/src/github.com/SevereOverfl0w/clj-resolve-tags master* 13s
❯ clj -Sdeps '{:deps {resolve-tags {:git/url "" :sha "44315f501e3b7fba40961563ba7fcfd0441225fa"}}}' -m resolve-tags not-deps.edn
Results in
{:paths ["src"]
;; cool comment
:deps
{potemkin {:mvn/version "0.1"}
tools.deps.alpha {:git/url ""
;; totally rad
:tag "tools.deps.alpha-0.5.460"
:sha "ca2aa1eb320687ce940a4f142c8d1af8f9a018bc"}
;; woah dude
tools.deps.alphax {:git/url ""
:tag "tools.deps.alpha-0.5.460"
;; zoom
:sha "ca2aa1eb320687ce940a4f142c8d1af8f9a018bc"}}}
#2018-10-2719:12andy.fingerhutIs there already an issue suggesting the addition of a "show version of clj/clojure and exit" option to clj and/or clojure? If not, would one be welcome? Seems useful in some debug interactions between people.#2018-10-2719:13dominicmIt does I think? -Sdescribe#2018-10-2719:16andy.fingerhutGot it. thx. Didn't realize that, and didn't see the more traditional -v or --version command line options.#2018-10-2719:17dominicmHave those been added? Neat. I haven't tracked really closely. #2018-10-2719:17andy.fingerhutI don't think so. I didn't see them.#2018-10-2719:22dominicmAh. I will stick to describe them 😊 #2018-10-2723:43jeff.terrellI've also been surprised by the lack of -v / --version options, FWIW.#2018-10-2805:10richiardiandreaI can trigger a (swap! verbose-surprise-count inc) as well :smiling_face_with_smiling_eyes_and_hand_covering_mouth: #2018-10-2904:00steveb8nWhen using :git/url deps, is there any way to emulate latest or snapshot style behaviour? I'm having to manually updates shas a lot.#2018-10-2906:36dominicmI don't think so - as an intentional decision.#2018-10-2906:37dominicmYou could write a script to update them automatically though.#2018-10-2908:13steveb8nInteresting idea. It seems like I won't be the only one wanting this. I wonder if there's another best/good practice to emulate maven in this way#2018-10-2910:13dominicmThe code is about 100 lines. You could modify the resolve-tags library I write quite easily. #2018-10-2910:16dominicmmodify the resolve-git-dep function with a hardcoded branch, basically https://github.com/SevereOverfl0w/clj-resolve-tags/blob/master/src/io/dominic/deps/resolve_tags.clj#2018-10-2910:33steveb8nHmm, maybe I should be writing a new “procurer”. I’ll look into that as a solution. thanks for the nudge in a new direction#2018-10-2906:38dominicmhttps://github.com/clojure/tools.gitlibs#revs supports branch names (e.g. master)#2018-10-2912:08alexmillerIt does, but tools.deps does not (gitlibs is intentionally broader and more generic)#2018-10-2909:36arrdemIf you're coordinating changes between many repos all the time I think it's worth considering whether the apparent decoupling you have at repo boundaries maps at all to stable designed API boundaries or whether you'd be better off admitting that you don't yet have decoupling and would be better served putting everything in one place for now.#2018-10-2910:31steveb8nGood point. In my case it’s not de-coupling I want, but shared specs. The specs are changing a lot currently, hence the need for regular updates#2018-10-2909:38arrdemIn slightly related news I got a Katamari v0.0.5 out the door which successfully does content addressing of build products (jars, uberjars, javac'd classfiles). At this point it's fully self-bootstrapped and stable as such. Incremental rebuilds were the big thing I wanted to get working before this week's meeup - if I can get repl and test tasks sorted out I'll be happy enough to really try and push this thing.
https://github.com/arrdem/katamari/releases/tag/v0.0.5 for your amusement.#2018-10-2910:58Christian JohansenI cleared out ~/.m2/repos/org/apache, and now I’m seeing this error with tools.deps:#2018-10-2910:58Christian JohansenException in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
#2018-10-2910:59Christian Johansenafter googling a little, it seems to go away with this in deps.edn:#2018-10-2910:59Christian Johansen:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2018-10-2910:59Christian Johansenwhat’s going on here?#2018-10-2911:33Christian Johansenbtw, I can observe that the jar in question is both being downloaded by the Clojure process, and is available in my ~/.m2 after its run (and failed)#2018-10-2912:10alexmillerThose maven repos are already set in the install deps.edn so that shouldn’t make much difference#2018-10-2912:10alexmillerCan you post the entire deps.edn for repro?#2018-10-2913:18jerger_at_ddaI had just the time to watch "Dependency Heaven" an like the arguments.
I'm working quite a lot with git dependencies on systems like gitbook (make books out of git-repos containing md) or smeagol (a git-repo / MD based wiki)
But I find it quite hard to find a good abstraction for addressing git resources.
Are there any plans to support other git servers like gitlab (the SaaS and all the on prem instances) or gitblit, gittea et al?
What about private repos?
In context of privisioning git repos I use this schema: https://github.com/DomainDrivenArchitecture/dda-git-crate/blob/master/doc/reference_domain.md
The relevant parts are Repository and GitCredential. These informations will lead to completely different urls like
>)deps.edn, asking them to add one.#2018-10-3016:41seancorfieldI've been trying to make sure I do that with every project I maintain.#2018-10-3016:43neural@seancorfield nice#2018-10-3016:43seancorfield(in fact, in the Contrib libraries I maintain, I've removed project.clj because it's misleading -- many people assume the version in that file, or the specific dependencies in that file, are definitive, whereas for Contrib projects, it's driven by pom.xml)#2018-10-3016:51neuralmaybe this is an other channel related question... but you guys know if clj-refactor/refactor-nrepl works with deps.edn???#2018-10-3017:07seancorfield@neural.works.com As far as I know, the most recent version of CIDER works with clj/`deps.edn` -- but maybe better to ask in #cider for more detail...?#2018-10-3017:08neural@seancorfield tks#2018-10-3019:43dominicmIt does. I set that up. #2018-10-3121:08timgilbertSay, anybody got something that works similarly to lein-try, where I can lein try clojure-lanterna and fire up a REPL with the latest version loaded?#2018-10-3121:11jeff.terrellRelative newbie here, but I think the -Sdeps option covers that use case, no?#2018-10-3121:12timgilbertI'm thinking in a bare directory with no existing deps.edn present#2018-10-3121:12ghadiyup, that's what it's for#2018-10-3121:12dominicmI would use find deps#2018-10-3121:13dominicmhttps://github.com/hagmonk/find-deps#2018-10-3121:15timgilbertThink that'll work, thanks @dominicm#2018-10-3121:46lilactownI use these bash aliases to try out new deps in a nice REPL: https://gist.github.com/Lokeh/a67bb2dc9295d7990d9f8525f478457c#2018-10-3121:50seancorfieldclj -Sdeps '{:deps {clojure-lanterna {:mvn/version "RELEASE"}}}' -- or you can use "LATEST" to look for snapshots instead.#2018-10-3121:50seancorfieldI use a command like that all the time when I just want to try out a library without any setup.#2018-10-3121:54seancorfieldThat looks like a fun library @timgilbert -- are you planning to build something public or is it just for fun?#2018-10-3123:02timgilbertJust for fun, was looking at some roguelike things from a HN post earlier today#2018-11-0101:42zaneHere's the post in case anyone else is interested: http://www.gamasutra.com/blogs/JoshGe/20181029/329512/How_to_Make_a_Roguelike.php#2018-11-0109:28borkdudeIs it possible to combine multiple R aliases into one alias?#2018-11-0109:42dominicmHow do you mean?#2018-11-0109:55borkdudeSay alias A says extra deps d1 d2, alias B says extra deps d3 d4, now I want to make an alias AB which combines A and B so it has extra deps d1 .. d4#2018-11-0109:57borkdudeBasically wondering if I could do something like https://github.com/borkdude/boot-bundle using deps.edn#2018-11-0110:07dominicm@borkdude there's no inheritance. So the only way to do it is by modifying the edn file. I've been writing https://github.com/SevereOverfl0w/ednup with the goal of it eventually supporting this.#2018-11-0116:50timgilbertSay, I'm trying to figure out how to effectively deal with the .cpcache directory in circleci. The way it works is that circleci can store a directory in a cache which is then available to subsequent steps (although the subsequent steps might themselves run on different VM instances).#2018-11-0116:50timgilbertSo what I currently do is run clojure -A:foo:bar -Spath to download a bunch of stuff and then cache the .m2 and .cpcache directories#2018-11-0116:51timgilbertMy question is, will tools.deps create a single .cpcache entry for the combined :foo:bar aliases, or will it create a :foo cache and a :bar cache? (Or all three?)#2018-11-0116:52timgilbertI'm wondering if it would be more efficient to run clojure -A:foo -Spath; clojure -A:bar -Spath#2018-11-0116:52alexmillercombined#2018-11-0116:52alexmillerthe cache key is based on the set of aliases used#2018-11-0116:52timgilbertAha, makes sense. Thanks @alexmiller#2018-11-0116:53alexmillerhttps://github.com/clojure/brew-install/blob/1.9.0/src/main/resources/clojure#L230-L237#2018-11-0117:35fmjreyHey, I'm migrating some lein project to deps.edn and somehow it looks like my dev/user.clj isn't loaded in the repl, even though I use clj -R:dev and the :dev alias does have :extra-paths ["dev" "fixtures"]. My user.clj is very simple:
(:require
[com.stuartsierra.component.user-helpers :refer [dev go reset]]))
clj -Stree -R:dev shows the component lib is loaded.
Still it's as if user.clj isn't loaded:
$ clj -R:dev dev/user.clj
$
No output or error.
$ clj -R:dev
Clojure 1.9.0
user=> (dev)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: dev in this context, compiling:(NO_SOURCE_PATH:1:1)
user=>
What could I be missing?#2018-11-0117:44hiredmanuse -A not -R#2018-11-0117:47fmjreyThanks, feel stupid now, wondering how I ended up using -R#2018-11-0117:49hiredmansimilar questions have come up a few times, it makes me wonder if there is a tutorial or documentation somewhere that is using -R in examples where it should be using -A#2018-11-0117:51fmjreyI think it must have been: https://clojure.org/guides/deps_and_cli#extra_deps#2018-11-0117:51fmjrey> You can add this dependency to your classpath by adding the :bench alias to modify the dependency resolution: clj -R:bench.#2018-11-0117:57fmjreyNot even sure I understand the difference between -A and -R from the cli doc#2018-11-0117:58fmjreyAhh this should explain it: https://clojure.org/reference/deps_and_cli#_aliases#2018-11-0117:51richiardiandreaquestion about CI and .m2 - is there anything I have to do to set Clojure CLI on a machine for being able to cache artifacts in .m2?#2018-11-0117:52richiardiandreathis machine does not seem to have Maven, do I need it? or .m2/settings.xml#2018-11-0117:52hiredmannope#2018-11-0117:53hiredmanif you are ci'ing inside some kind of container you might want to set things up so ~/.m2 is stored in a way that persists across different runs#2018-11-0118:42richiardiandreais .m2 created for me?#2018-11-0118:43hiredmanyes#2018-11-0118:43hiredmantools.deps will store git deps in ~/.gitlibs too#2018-11-0118:46richiardiandreaok thanks, my CI executes things in a weird way I think...every call to clojure resets .m2 so it downloads everything again, investigating...#2018-11-0118:48hiredmanyeah, you are likely building in some kind of container environment with nothing persisted between builds#2018-11-0118:48richiardiandreaI am on Azure Pipeline..#2018-11-0118:48richiardiandreayeah..#2018-11-0118:50hiredmanin theory that should be fine, m2 is a cache, so if the cache is missing it will just rebuild it, but it is much better to persist m2#2018-11-0118:50hiredmanyou actually might be running in to another caching issue too#2018-11-0118:50richiardiandreayeah I see it is redownloading the whole thing#2018-11-0118:51hiredmanthe clj tool fetches deps and builds the classpath once and sort of caches that, if the deps it fetches go missing, but the cache file it generates is still there, it will blow up#2018-11-0118:51richiardiandreamakes sense yeah#2018-11-0118:51richiardiandreathank you so much#2018-11-0119:45richiardiandreawhich command creates .m2? I know first execution does, but I would like to see it there before running my actual other tasks#2018-11-0119:45richiardiandreacan I just do clojure -Sdescribe?#2018-11-0119:45richiardiandreaactually let me try locally#2018-11-0119:46richiardiandreayep that works#2018-11-0119:46richiardiandrea$ mv .m2 .m2-backup
$ clojure -Sdescribe
Downloading: org/clojure/clojure/1.10.0-alpha6/clojure-1.10.0-alpha6.pom from
Downloading: org/clojure/spec.alpha/0.2.168/spec.alpha-0.2.168.pom from
Downloading: org/clojure/pom.contrib/0.2.2/pom.contrib-0.2.2.pom from
Downloading: org/clojure/core.specs.alpha/0.2.36/core.specs.alpha-0.2.36.pom from
Downloading: org/clojure/clojure/1.10.0-alpha6/clojure-1.10.0-alpha6.jar from
Downloading: org/clojure/spec.alpha/0.2.168/spec.alpha-0.2.168.jar from
Downloading: org/clojure/core.specs.alpha/0.2.36/core.specs.alpha-0.2.36.jar from
{:version "1.9.0.397"
:config-files ["/home/arichiardi/.local/lib/clojure/deps.edn" "/home/arichiardi/.clojure/deps.edn" ]
:install-dir "/home/arichiardi/.local/lib/clojure"
:config-dir "/home/arichiardi/.clojure"
:cache-dir "/home/arichiardi/.clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
$ ll ~/.m2
total 24
drwxrwxr-x 3 arichiardi arichiardi 4096 Nov 1 12:45 ./
drwxr-xr-x 120 arichiardi arichiardi 16384 Nov 1 12:45 ../
drwxrwxr-x 3 arichiardi arichiardi 4096 Nov 1 12:45 repository/
#2018-11-0119:49richiardiandreait is downloading alpha6 by default#2018-11-0120:24alexmillerNo, it’s not - that’s an artifact of your deps.edn files#2018-11-0120:27alexmillerBy default, clj will use 1.9.0 based on the ~/.clojure/deps.edn#2018-11-0120:46richiardiandrearight#2018-11-0120:47richiardiandreaI should have use -Srepo maybe?#2018-11-0120:47richiardiandreaanother question I have is if I can have aliases in parent dir and consume in children dir, in a monorepo setup for instance#2018-11-0121:05richiardiandreaI tried with CLJ_CONFIG to the root but I don't see it working...#2018-11-0121:06richiardiandreaoh now I see it working cool#2018-11-0201:03richiardiandreaHow long does builds take for you folks in average? I am being told node builds take 5-10 minutes and I want to make sure I am not doing anything wrong before answering. At the moment a monorepo build takes 22 minutes and it downloads ~50 deps#2018-11-0204:06richiardiandreaThis is more of a #clojurescript question actually#2018-11-0207:42slipsetFirst of all, I know tools.deps/clj is not meant as a build tool. But still:
We now have a bunch of utils like pack, Eastwood, depot, testrunner, etc which are useful tools to use in a build pipeline.
I know I can add aliases to all of these in my home-directory’s deps.edn, but that doesn’t really work across multiple machines.
Having all the aliases duplicated across multiple projects doesn’t seem like a great solution either.
One solution I could think of is that deps.edn could refer to other deps.edns and use their aliases. Would that be possible?#2018-11-0209:19Christian Johansenwhat exactly is the problem in duplicating between projects? lack of standardization?#2018-11-0209:20Christian JohansenI have stuff like this:#2018-11-0209:20Christian Johansen:ancient {:extra-deps {olical/depot {:mvn/version "1.2.0"}}
:main-opts ["-m" "depot.outdated.main"]}
#2018-11-0209:21Christian Johansenyou want to make that implicit in your deps.edn?#2018-11-0209:39slipsetI was imagining that when I wanted to bump the version on olical/depot, I’d have to check out all my projects and bumpt it.#2018-11-0209:39slipsetAlso, some of the aliases are somewhat more involved than this.#2018-11-0209:41Christian Johansenyup. it’s a difficult trade-off. it’s tempting to standardize or “dedupe” some of this boilerplate, but that creates problems when things aren’t as equal anymore#2018-11-0209:42Christian Johansene.g. you loose the flexibility#2018-11-0209:43slipsetI’ve come to realise this. But that’s another discussion we could have over a cup of coffee IRL 🙂#2018-11-0209:43Christian Johansenif you have a lot of very similar projects I guess you could make your own dev package, and encode the aliases in it, and then use it like so:#2018-11-0209:43Christian Johansen{:aliases
:dev {:extra-deps {my.company/my-bundle {:mvn/version "1234"}}}
:test {:main-opts ["my.bundle.test"]}
:ancient {:main-opts ["my.bundle.ancient"]}
:build {:main-opts ["my.bundle.build"]}}
#2018-11-0209:44Christian Johansenyes, for sure 🙂#2018-11-0209:45Christian Johansenbut that approach does quickly leads to entangled framework world, use with caution#2018-11-0209:48slipsetI guess so.#2018-11-0209:50Christian JohansenI don’t have enough similar projects to warrant the effort, but this is an approach I would consider to contain the duplication and lightly enforce some standardization. you’d still have to bump the shared dependency in all projects, but at least you reduced n deps/aliases to 1#2018-11-0209:52Christian JohansenI’m sure there are other ways to approach this as well#2018-11-0210:02orestisOne thing I noticed really is that when most of those things are just standalone utilities, they shouldn’t even be in my main deps.edn file, because they don’t need all my “production” stuff to run.#2018-11-0212:34alexmillerExactly - put them in your ~/.clojure/deps.edn#2018-11-0213:29orestisIs there then a way for the clj to ignore any local deps.edn?#2018-11-0213:30orestisMeaning, when the Classpath is created, it only has the stuff from ~/.clojure/deps.edn and nothing from a local deps.edn file.#2018-11-0213:48alexmillerno, why would you want that?#2018-11-0214:13orestisIn the context of a standalone library, e.g. clj-new, cljfmt etc etc that does “build tool” stuff — it has no need to bring in the application-level deps. I’m not sure what the ramifications of having more things in the classpath are though — apart from the potential version conflicts which I think are addressable right now.#2018-11-0210:03orestisI’m always thinking that perhaps it’s time for a new “dev-time” tool that provides a wrapper and some porcelain around those single-purpose libraries. Sure you lose the flexibility, but you also don’t have to make all those decisions when you are a newcomer to the ecosystem, plus the single-purpose libraries can still be used standalone and can evolve at their own pace.#2018-11-0210:09Christian Johansenwherever it is, all the deps and scripts required to build and deploy the app should reside in the repo at least#2018-11-0210:09dominicm@orestis would such a thing be put into each project? Or be part of the setup guide for ~/.clojure/deps.edn when you're new to a project?#2018-11-0210:11Christian JohansenI strongly believe that developing any app should require as few pre-installed tools as possible, and that the code/repo should be as self-contained as possible#2018-11-0212:27rickmoynihanI agree with this sentiment too. tools.deps is quite good here, as you can effectively ship tooling with a repo; and version lock the tools - without having to include binaries etc.#2018-11-0210:11orestisI think it could be its own standalone CLI, like lein or boot.#2018-11-0210:12orestisBut in the end all it would do is generate the correct clj/clojure invocation for you.#2018-11-0210:13Christian JohansenI’m currently using deps.edn + Makefile to get the job done. I see some things that will probably seem like tedious repetition at some point, but so far I’m loving the simplicity of it. no magic, you can see everything#2018-11-0210:13orestisI see your point though @christian767 — though with tools.deps you need to have clj installed — otherwise, well, you have to fallback to java -cp ..., no?#2018-11-0210:14Christian Johansenyes, I think clj and java is a good baseline requirement for clojure projects at this point#2018-11-0210:14orestisRight, a more opinionated Makefile is what I’m trying to describe.#2018-11-0210:14Christian Johansensure#2018-11-0210:14orestisMy experience in trying to setup a new deps.edn based project was a few days trying out the various uberjar libraries, then a linter, then a formatter, then an outdated-deps-finder etc etc etc.#2018-11-0210:15orestisI think the community would benefit if this is done for you, while still maintaining the much-needed transparency and zero magic.#2018-11-0210:15Christian JohansenI agree#2018-11-0210:16orestisI wonder how could this move forward? Ask people to upload their deps.edn and makefiles and shell scripts somewhere and compare approaches? I’m too much of a newb to do this based only on my assumptions, and different projects will have wildly different needs.#2018-11-0210:18dominicm@orestis JUXT are solving this with https://github.com/juxt/edge#2018-11-0210:19dominicmYou start a project by cloning edge, and then modifying it. Everything is set up for you. clj is all you need.#2018-11-0210:19dominicmTemplates also solve this problem.#2018-11-0210:20Christian Johansenbasically, scaffolding#2018-11-0210:21dominicmI wouldn't say scaffolding is a good comparison. It's more of a cookie cutter or starting point than anything.#2018-11-0210:21orestisThe issue with templates/scaffolding is that after the initial project creation, you are on your own.#2018-11-0210:21Christian Johansenmy issue with this approach is that it sets you off for an overwhelming start, and it is too easy to end up with a bunch of stuff laying around that you don’t really need#2018-11-0210:21dominicm@orestis that's why we suggest cloning. You can upgrade by doing git merge edge/master#2018-11-0210:22dominicmI've been making some adjustments to reduce the chances of conflicts to very low. It should only happen if you modify things (in which case, what do you expect?)#2018-11-0210:22dominicmThe documentation around this isn't great, but it's how we expect edge to be used, and how we use it internally.#2018-11-0210:24dominicmblugh, s/modify things/modify shared areas in a separate folder/#2018-11-0210:26orestisI see your point — and it might be hugely valuable for a consulting company, since you tend to start projects very often, right? I’m more after something that decouples some very common concerns for most people, while imposing no opinions on how your actual app is structured. Meaning, I just want to run a command to give me outdated dependencies, a command to run all my tests, a command to build my uberjar. I’d like newcomers to the community to have a single entry point, and if/when they want more power, be able to drop down to the layer below.#2018-11-0210:27orestisCrucially, they shouldn’t have to know the underlying libraries if they don’t care. Similarly to how you don’t really know how lein builds uberjars.#2018-11-0210:27dominicmI know 🙈 You don't want to#2018-11-0210:27orestisAnd also crucially though, the uberjar building library is standalone and people can use it as they see fit.#2018-11-0210:27Christian Johansenkoacha does this for tests#2018-11-0210:28Christian Johansenyou could create a binstub on your box, run koacha in your app, and it’ll “just work”#2018-11-0210:28dominicmWe start, and maintain, a lot of projects, yep 🙂.
edge is slightly more opinionated than that, it assumes integrant for example. But I think the implementation would work for this? Use a git repo, and merge periodically to have the uberjar builder updated by maintainers upstream.#2018-11-0210:29orestisExactly @christian767, kaocha is very good at that, it has its own CLI - like thing, but you can also use its API if you want to.#2018-11-0210:30Christian Johansenyup#2018-11-0210:31dominicmBinstubs are only part of the puzzle though 🙂
What would provide:
> I think the community would benefit if this is done for you, while still maintaining the much-needed transparency and zero magic.
?#2018-11-0210:31orestisSo what I’m after is — a cli tool that you could install, then run XXX test and it would do the right thing, using kaocha under the hood. Kaocha then can keep evolving and people can use it standalone without the XXX tool.#2018-11-0210:32orestisYou wouldn’t know that kaocha is used unless you really want to — kaocha is a very good example since it plays well with clojure.test. Uberjars can be more fiddly to configure, I guess?#2018-11-0210:33slipsetThis is sort of the discussion I was looking for#2018-11-0210:33slipsetBut in one way, I guess we’re discussing yet another lein or boot#2018-11-0210:33dominicmjars need a lot of configuration in my experience 🙂 Clients come up with all sorts of novel environments in which we have to deploy.#2018-11-0210:33dominicm@slipset I wouldn't like to see another lein or boot fwiw.#2018-11-0210:34slipsetI know, but the ideas I’m sort of looking for very quickly lead in that direction, unfortunatly#2018-11-0210:35slipsetEven if it’s only @christian767 Makefile or Juxt’s edge.#2018-11-0210:35slipsetI could imagine ending up in:#2018-11-0210:36slipset$ make -f /usr/local/etc/default-makefile-for-app setup#2018-11-0210:36slipsetwhich is basically lein new app or clj -A:clj-new app ...#2018-11-0210:36slipsetor some sort.#2018-11-0210:38orestisI’m not 100% sure if this only an “ergonomics” thing — or if some plumbing is still missing from tools.deps.#2018-11-0210:39dominicmGiven Stu's advocacy for CLJ-2373 (I've memorized it he's said it so much!), let's take a leaf out of the Clojure/core approach to solving problems.
What is the problem statement here?#2018-11-0210:40dominicmIf we were only trying to solve the assembly problem, then templates are fine. So there's another element too I'm guessing?#2018-11-0210:46dominicmI kinda get the impression we're each looking for different additional properties. I've seen mention of "project-local isolation", "baseline as clj", and "not caring about underlying library".#2018-11-0211:02dominicmfwiw, I care more about "baseline as clj" because it's the minimum I can expect anyone to have done, ever. If we're doing training as a company, getting binstubs installed, covering all the distros is more hassle than it's worth for me.
I would probably find it acceptable if a project had some kind of alias like clj -A:tool-name which would then do the rest for itself. add-lib is pretty compelling here, as the tool can dynamically fetch it's dependencies as needed.#2018-11-0306:02seancorfield@alexmiller merged master back to the add-lib branch and we just added a :deps alias at work to that SHA so we can use add-lib in our REPLs. It's nice. I need update my dot-clojure file/repo with that too.#2018-11-0212:13orestisNice breakdown. Fully agree.#2018-11-0212:24orestisPossibly relevant: https://www.reddit.com/r/Clojure/comments/9tj6d1/katamari_a_clojure_build_tool_a_talk_by_reid/#2018-11-0216:00seancorfieldI uploaded my https://github.com/seancorfield/dot-clojure as a guide for others some time ago -- I think sharing these is useful while the tool is still new. #2018-11-0216:04seancorfieldAs for a self-contained project: yes, very important -- it's why we have a shell wrapper that sets CLJ_CONFIG (to point to a repo-specific "dot Clojure" folder) and we have a deps.edn file for each subpar object in the repo. #2018-11-0216:26richiardiandreaFunny I was doing the same yesterday but it seemed like a hack ^#2018-11-0216:27andy.fingerhutBut today, it no longer seems like you are hacking. Today, you are corfielding 🙂#2018-11-0216:28richiardiandreaIt seems like everybody is going to corfield their deps.edn sooner or later 😄#2018-11-0216:29andy.fingerhutSean, if this goes seriously weird, I sincerely apologize#2018-11-0216:44andy.fingerhutIf it only goes slightly weird, then .... you are welcome? 🙂#2018-11-0218:07seancorfieldWeird is good. I like weird. :rolling_on_the_floor_laughing:#2018-11-0218:10seancorfieldBack to the CLJ_CONFIG stuff (I was on my phone earlier)... we've found the ability to specify a "home" deps.edn via CLJ_CONFIG is great! It means a project can have a "home" configuration for aliases that provide project-specific tooling as well as :override-deps (which we rely on heavily to "pin" versions across subprojects within the overall project.#2018-11-0218:13seancorfieldI talked a bit our shell wrapper a week or two back I think. Mostly it takes commands of the form build alias(es) subproject and it goes into that subproject and runs CLJ_CONFIG=../versions clojure -A:defaults:aliases (edited to add our #2018-11-0218:19alexmillerIs that only for aliases?#2018-11-0218:19seancorfieldIt does a few mini-DSL-y sort of things around plural aliases or aliases with certain prefix/suffix strings as well as expanding some internal "aliases" but it's only a few tens of lines of shell script.#2018-11-0218:20seancorfield@alexmiller We have the "pinned" version of Clojure specified as a regular :deps in that versions/deps.edn file but everything else is just aliases I think.#2018-11-0218:23seancorfieldOh, and it adds our Archiva Maven repo as well.#2018-11-0218:27seancorfieldIt's definitely shown the need for all deps-based tooling to support correct deps.edn merging with support for :override-deps and aliases, if it doesn't just execute in the context of your project's classpath (Depot, for example, already supports aliases but needed a PR to support :override-deps merging).#2018-11-0218:27seancorfieldIt would definitely simplify usage if :override-deps could be top-level and not need an alias.#2018-11-0219:06dominicm@seancorfield fwiw, I don't like the CLJ_CONFIG solution because it takes away the ability to have aliases (e.g. A custom repl) from a user.
I understand your choice and think it is pragmatic though. #2018-11-0219:40seancorfieldYeah, it's a compromise. We have REPL-related aliases in that deps file, as well as other tooling.#2018-11-0219:41seancorfieldWhat I'd really like would be the ability to specify an extra deps.edn file easily between user-deps and project-deps.#2018-11-0223:36seancorfield@alexmiller Is this something you'd entertain as an enhancement? Essentially to support something like <system>/deps.edn, <user>/deps.edn, <tooling>/deps.edn, (project) deps.edn.#2018-11-0223:37seancorfieldI wasn't sure if you just missed this in the flow of questions in the channel since you didn't comment -- and I sort of expected you to 🙂#2018-11-0300:04alexmillerDidn’t see it#2018-11-0300:05alexmillerDon’t know - have been hoping to move towards fewer deps.edn files, not more :)#2018-11-0300:08seancorfieldFair enough. The underlying t.d.a. lib supports an arbitrary number of deps.edn files tho'... and it would be nice and generic to surface that in clj/`clojure` 🙂#2018-11-0300:09seancorfieldYou can easily merge extra stuff at the end with -Sdeps but you can't insert anything before the project deps.edn which is where I find the need -- and I think I'm not alone.#2018-11-0300:10seancorfieldWe can get away with "replacing" the user-deps via CLJ_CONFIG for our needs but it does mean some duplication and inconvenience.#2018-11-0300:28alexmillerOther things are pointing away from having an arbitrary number of deps so that’s much more likely to die#2018-11-0300:28alexmillerBut I asked because I’m interested in the use cases#2018-11-0300:28seancorfieldInteresting. I wonder how we'll make things work if that goes away. It's kind of fundamental to our workflow right now.#2018-11-0300:32seancorfieldI guess we'd need to maintain a variant of the clojure script that allowed us to specify an extra deps file around these lines # Chain deps.edn in config paths. repro=skip config dir
if "$repro"; then
config_paths=("$install_dir/deps.edn" "deps.edn")
else
config_paths=("$install_dir/deps.edn" "$config_dir/deps.edn" "deps.edn")
fi
config_str=$(printf ",%s" "${config_paths[@]}")
config_str=${config_str:1}
#2018-11-0300:34seancorfield(which part are you saying might "die"? support for CLJ_CONFIG or support for the --config-files command line argument in t.d.a.?)#2018-11-0219:41seancorfieldSo we'd have <system>/deps.edn, <user>/deps.edn, <tooling>/deps.edn, (project) deps.edn.#2018-11-0219:42timgilbertSay, I'm getting this error trying to mvn install a tools.deps.alpha repo I've got checked out locally, anyone know if there is some special stuff I need to do in order to build it?
[INFO] --- clojure-maven-plugin:1.7.1:test (clojure-test) @ tools.deps.alpha ---
Error: Could not find or load main class clojure.main
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
#2018-11-0219:46alexmillerCan you provide more info? Is there a pom? Did you gen it or make it? What do you mean by a “tools.deps.alpha repo”?#2018-11-0219:47timgilbertSure. I just cloned https://github.com/clojure/tools.deps.alpha and ran mvn install. There is a pom, which I've left unchanged from the repo. I couldn't find any docs about a build process, but I'm sure I could have missed something.#2018-11-0219:49timgilbert(I get the same error running mvn test, which makes sense)#2018-11-0219:50timgilbertHere is the full output:
% mvn install
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 1 modules...
[INFO] Installing Nexus Staging features:
[INFO] ... total of 1 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building tools.deps.alpha 0.5.461-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-source (add-clojure-source-dirs) @ tools.deps.alpha ---
[INFO] Source directory: /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/main/clojure added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-resource (add-clojure-source-dirs) @ tools.deps.alpha ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-test-source (add-clojure-test-source-dirs) @ tools.deps.alpha ---
[INFO] Test Source directory: /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/test/clojure added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-test-resource (add-clojure-test-source-dirs) @ tools.deps.alpha ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ tools.deps.alpha ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/main/resources
[INFO] Copying 18 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.0:compile (default-compile) @ tools.deps.alpha ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ tools.deps.alpha ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.0:testCompile (default-testCompile) @ tools.deps.alpha ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ tools.deps.alpha ---
[INFO]
[INFO] --- clojure-maven-plugin:1.7.1:test (clojure-test) @ tools.deps.alpha ---
Error: Could not find or load main class clojure.main
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.073 s
[INFO] Finished at: 2018-11-02T19:48:28+00:00
[INFO] Final Memory: 19M/295M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.7.1:test (clojure-test) on project tools.deps.alpha: Clojure failed. -> [Help 1]
#2018-11-0219:51timgilbertSorry, I could put this all in a ticket too. Just thought I might be missing something obvious.#2018-11-0219:52alexmillerOk, so you actually mean tda#2018-11-0219:52alexmillerNo magic here, it is just a maven build and I run mvn test on it all the time#2018-11-0219:52alexmillerWhat java version?#2018-11-0219:53alexmillerProb needs 8+#2018-11-0219:57timgilbert% java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-1ubuntu0.16.04.1-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
% mvn --version
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_181, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-17134-microsoft", arch: "amd64", family: "unix"
#2018-11-0219:57timgilbertThis is running under ubuntu in the "windows subsystem for linux" which is a bit of a weird environment but all the usual clojure stuff works fine#2018-11-0219:57alexmillerI’m using Java 11 and Maven 3.5.4 at the moment, but I don’t have any reason to believe your versions are not sufficient#2018-11-0219:58alexmillerthe build box is certainly using much older stuff to build and deploy that lib#2018-11-0219:58alexmillerworks on my machine :)#2018-11-0219:58alexmillerand the build box :)#2018-11-0219:58alexmillertry throwing -X -e on your mvn command#2018-11-0220:00timgilbertI was wondering if I might need to build clojure locally first or something#2018-11-0220:00alexmillernope#2018-11-0220:01alexmillerit’s just using the released clojure 1.9.0#2018-11-0220:01timgilbertHere's the exception with -X -e:
[ERROR] Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.7.1:test (clojure-test) on project tools.deps.alpha: Clojure failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.7.1:test (clojure-test) on project tools.deps.alpha: Clojure failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Clojure failed.
at com.theoryinpractise.clojure.AbstractClojureCompilerMojo.callClojureWith(AbstractClojureCompilerMojo.java:464)
at com.theoryinpractise.clojure.AbstractClojureCompilerMojo.callClojureWith(AbstractClojureCompilerMojo.java:366)
at com.theoryinpractise.clojure.ClojureRunTestWithJUnitMojo.execute(ClojureRunTestWithJUnitMojo.java:138)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
#2018-11-0220:02alexmillerIt’s failing in the test step, which is kinda weird. You could use -Dmaven.test.skip=true if you wanted to just move past that#2018-11-0220:03timgilbertAha! That did it.#2018-11-0220:03alexmillerit’s failing to run Clojure to run the tests. not sure why that would be#2018-11-0220:04alexmillerI’ve seen a lot of maven build failures, but I haven’t seen this one :)#2018-11-0220:04timgilbertMy next question was going to be how to run the tests 😉#2018-11-0220:04timgilbertIt's possible that some local modifications I made messed something up, let me try on a clean master#2018-11-0220:05alexmillerhere’s the build box running it: https://build.clojure.org/job/tools.deps.alpha/153/console (Java 8 / Maven 3.2.5)#2018-11-0220:09timgilbertHmm, still getting the same thing on a clean master. I'll see if I can repo at home on my mac and file a ticket if so. Thanks for looking.#2018-11-0220:17seancorfieldIt works on my Mac. I haven't tried to build/install any Clojure Contrib projects on WSL on my Windows 10 laptop. I'll try to remember to test that later today.#2018-11-0220:55timgilbertThe root of the problem seems to be that I'm getting the clojure dependency with scope provided.
% mvn dependency:tree | grep org.clojure
[INFO] org.clojure:tools.deps.alpha:jar:0.5.461-SNAPSHOT
[INFO] +- org.clojure:clojure:jar:1.9.0:provided
[INFO] | +- org.clojure:spec.alpha:jar:0.1.143:provided
[INFO] | \- org.clojure:core.specs.alpha:jar:0.1.24:provided
[INFO] +- org.clojure:data.xml:jar:0.2.0-alpha5:compile
[INFO] | \- org.clojure:data.codec:jar:0.1.0:compile
[INFO] +- org.clojure:tools.gitlibs:jar:0.2.64:compile
[INFO] \- org.clojure:tools.cli:jar:0.3.5:compile
#2018-11-0220:56timgilbertAnyways, I'll mess around a bit more and put a ticket up. I'm guessing this is something local to my machine.#2018-11-0220:56alexmillerinteresting. I wonder why that’s different here vs there#2018-11-0221:12seancorfield@U08QZ7Y5S FYI, if you're relying on WSL, avoid Insider build 18272 -- they broke WSL in that build (they know about it, not sure when a fix will flight tho').#2018-11-0221:13timgilbertAfter the October fiasco, I'm avoiding Windows updates altogether for some time#2018-11-0221:13alexmillerOctober Fiasco is my new hardcore band btw#2018-11-0221:20seancorfieldI'm on Fast Ring Insider builds so my WSL is bricked on both my Windows machines. Fortunately, I still have my Mac for dev work! 🙂#2018-11-0717:02timgilbertFWIW, filed https://dev.clojure.org/jira/browse/TDEPS-105 to track this mishegas. My home mac is sad so I haven't had time to test it there#2018-11-0717:20alexmiller@U08QZ7Y5S I believe you’re experiencing this, and I feel bad about that, but priority-wise has to fall into my “won’t fix” bucket, so I’m going to decline it. If you manage to find a fix, happy to see it re-opened and consider.#2018-11-0717:25timgilbertThat's totally fine with me, I mostly just wanted to get it documented in case somebody else was seeing it. I have to confess that I have no idea how mvn test ever works without a concrete clojure.jar in the dependency tree, though.#2018-11-0717:33alexmillerprob via the clojure maven plugin (which is in the classpath)#2018-11-0717:44timgilbertI thought of that, but it's not in there: https://github.com/talios/clojure-maven-plugin/blob/clojure-maven-plugin-1.7.1/pom.xml#2018-11-0219:44johnWhat are people using as an environ replacement? Does it only work with a project.clj?#2018-11-0219:47zanejuxt/edge is deps.edn-based and uses Aero: https://github.com/juxt/aero#2018-11-0219:47seancorfieldWe have our own, internal, configuration library that can use env vars but mostly relies on EDN files.#2018-11-0223:05dominicmI'd enjoy evangelizing aero to you 🙂 Any big differences between aero and your system?#2018-11-0223:34seancorfieldI'd have to go look at Aero again. We did a review of pretty much everything available out there when we decided to "roll our own" a while back...#2018-11-0311:09yogidevbear(whispers "aero" into Sean's ear while he's sleeping)#2018-11-0219:48seancorfieldWe've considered open sourcing it, but it does contain some proprietary stuff that we haven't had time (or inclination) to refactor out yet.#2018-11-0220:04kennyIf I have this in my :deps:
compute/integrations {:git/url "
And I run clj -Stree -Sforce, I get this:
compute/integrations
Why is commons-logging/commons-logging included?#2018-11-0220:07alexmillerdunno, that’s weird#2018-11-0220:07alexmillerfeel free to file a ticket with a repro if you can slim it down#2018-11-0220:08alexmillerit being under a git dep might be part of the answer, not sure#2018-11-0220:08kennyWill see if I can get a repro.#2018-11-0220:51kennyFirst cut at the obvious repro case and I cannot reproduce. I made a git repo with a deps.edn that has clj-http as the dep. I made directory with a deps.edn on my computer that added that lib as a dep with the commons-logging exclusion and it behaved correctly. Is there something else I should be looking for in my current problem case that may shed some light here?#2018-11-0220:53kennyIf I add that exact dependency to my new project:
{:deps {compute/integrations {:git/url "
commons-logging is not in the tree, as expected.#2018-11-0220:55kenny#2018-11-0221:00kennyThis dep excludes httpclient, as expected.
compute/integrations {:git/url "#2018-11-0221:01kennyIs there a way to see all the deps that have commons-logging as a dep?#2018-11-0221:03kennyRemoving all other deps in my project besides Clojure and compute/integrations causes the exclusion to work correctly.#2018-11-0221:39kennyGot a repro. There is something seriously funky going on. Repro is here: https://github.com/kennyjwilli/clj-exclusions. cd into the app directory and run clj -Stree -Sforce. I get this:
org.clojure/clojure 1.9.0
org.clojure/core.specs.alpha 0.1.24
org.clojure/spec.alpha 0.1.143
integrations/integrations ..
clj-http/clj-http 3.9.1
commons-codec/commons-codec 1.11
org.apache.httpcomponents/httpasyncclient 4.1.3
org.apache.httpcomponents/httpcore-nio 4.4.6
slingshot/slingshot 0.12.2
commons-io/commons-io 2.6
org.apache.httpcomponents/httpcore 4.4.9
org.apache.httpcomponents/httpclient-cache 4.5.5
org.apache.httpcomponents/httpclient 4.5.5
commons-logging/commons-logging 1.2
potemkin/potemkin 0.4.5
clj-tuple/clj-tuple 0.2.2
riddley/riddley 0.1.12
org.apache.httpcomponents/httpmime 4.5.5
com.amazonaws/aws-java-sdk-cloudwatch 1.11.399
com.amazonaws/jmespath-java 1.11.399
com.fasterxml.jackson.core/jackson-databind 2.6.7.1
com.fasterxml.jackson.core/jackson-core 2.6.7
com.fasterxml.jackson.core/jackson-annotations 2.6.0
com.amazonaws/aws-java-sdk-core 1.11.399
joda-time/joda-time 2.8.1
com.fasterxml.jackson.dataformat/jackson-dataformat-cbor 2.6.7
software.amazon.ion/ion-java 1.0.2
Now if I change the name of the dependency from integrations to lib, I get this output from clj -Stree -Sforce:
org.clojure/clojure 1.9.0
org.clojure/core.specs.alpha 0.1.24
org.clojure/spec.alpha 0.1.143
com.amazonaws/aws-java-sdk-cloudwatch 1.11.399
com.amazonaws/jmespath-java 1.11.399
com.fasterxml.jackson.core/jackson-databind 2.6.7.1
com.fasterxml.jackson.core/jackson-core 2.6.7
com.fasterxml.jackson.core/jackson-annotations 2.6.0
com.amazonaws/aws-java-sdk-core 1.11.399
joda-time/joda-time 2.8.1
com.fasterxml.jackson.dataformat/jackson-dataformat-cbor 2.6.7
software.amazon.ion/ion-java 1.0.2
org.apache.httpcomponents/httpclient 4.5.5
commons-logging/commons-logging 1.2
lib/lib ..
clj-http/clj-http 3.9.1
commons-codec/commons-codec 1.11
org.apache.httpcomponents/httpasyncclient 4.1.3
org.apache.httpcomponents/httpcore-nio 4.4.6
slingshot/slingshot 0.12.2
commons-io/commons-io 2.6
org.apache.httpcomponents/httpcore 4.4.9
org.apache.httpcomponents/httpclient-cache 4.5.5
potemkin/potemkin 0.4.5
clj-tuple/clj-tuple 0.2.2
riddley/riddley 0.1.12
org.apache.httpcomponents/httpmime 4.5.5
It is different and all I did was change the name. What is going on here?#2018-11-0221:52alexmillerthe ordering is somewhat dependent on traversal order as to which one is encountered first, but you shouldn’t be getting a dep that is excluded#2018-11-0221:53alexmillerthere are a couple tickets out there that I’ve looked at and there are a couple issues to resolve - it’s likely this is the same as one of them#2018-11-0221:54kennyOk. Which tickets?#2018-11-0221:55seancorfieldThey're only different because org.apache.httpcomponents/httpclient is coming in via two dependencies, not just one.#2018-11-0221:56seancorfieldIn the integrations case, httpclient and logging come in via clj-http. In the lib case, httpclient and logging are coming in via aws cloudwatch (via aws core)#2018-11-0221:57kennyThe inconsistency is quite misleading though.#2018-11-0221:58alexmillerTDEPS-58 (and TDEPS-79)#2018-11-0221:59kennyYep that sounds like it.#2018-11-0221:58seancorfieldIf you have the exclusion on clj-http, it won't stop it being pulled in if the cloudwatch dep is walked first -- since it won't have seen the exclusion at that point.#2018-11-0221:59seancorfieldWe've had to be very aggressive with exclusions to stop up all the "holes" where certain things leak in...#2018-11-0221:59alexmillerthere have been some requests for “global” exclusions#2018-11-0221:59kennyAggressive meaning exclusions on a lot of your deps?#2018-11-0222:00alexmillerI’m not necessarily opposed to that, but haven’t thought through all the implications#2018-11-0222:00kennyI think Boot has that.#2018-11-0222:00seancorfieldYes. We've had to add the same/similar exclusions on several deps across our subprojects in order to completely "hide" them.#2018-11-0222:01kennyWould global exclusions solve your problem?#2018-11-0222:01seancorfieldWe've used global exclusions with Boot. It's a bit of a sledgehammer. We used it to exclude Clojure and ClojureScript across everything.#2018-11-0222:03seancorfieldYes, global exclusions would "solve" the problem but it's really a huge hammer to crack a little nut. And the fall out is that you then need to ensure you bring in all those things explicitly in the right subprojects -- which is a pain.#2018-11-0222:03alexmillerseems like that would make it hard to use Clojure or ClojureScript :)#2018-11-0222:03seancorfieldYou can bring them in at the top-level. It just ensures nothing else brings them in.#2018-11-0222:05alexmillerso I’d say that’s global-except-at-top-level :)#2018-11-0222:06seancorfieldYeah, well, Boot is nothing if not "pragmatic" 🙂#2018-11-0222:07alexmillerI’m just saying, that’s a subtlety#2018-11-0222:03kennyDependency resolution wouldn't solve that for you?#2018-11-0222:06alexmillerwith Maven dependency, no#2018-11-0222:06alexmillerMaven takes the first version it encounters#2018-11-0222:06seancorfieldTwo things: 1. we wanted ClojureScript excluded completely for all our backend projects, 2. we run "pedantic" style checks (or, at least, we did in lein and boot) so if potentially conflicting versions came in, we would get an exception -- and we didn't want to add those exclusions to nearly every Clojure dependency, since everything depends on different versions of Clojure out there 🙂#2018-11-0222:11kennyAnyway, thanks for the help. Excluding commons-logging from my other top level AWS deps fixed the issue. I voted and followed TDEPS-58.#2018-11-0300:24lwhortoni want to report a big-ol THANK YOU to the hard work behind tools-deps. i just started a greenfield clojure project and was able to create an nrepl “reloaded” workflow using only 1. deps.edn file 2. makefile for convenience 3. cat. no more boot, no more lein (for now, still have to look at generating jars).#2018-11-0300:24lwhortonis this what cleanliness feels like? have i found it?! troll#2018-11-0300:25alexmillerLiving the dream#2018-11-0300:28seancorfieldI'll second that THANK YOU since we have essentially eliminated Boot from our primary Clojure workflow across all tiers (dev through QA through production) and we like the simplicity of it all.#2018-11-0310:28borkdude@lwhorton mind sharing your deps.edn? I’m interested in enabling nREPL for a project so I don’t have to use lein#2018-11-0316:52lwhortonsure:
:aliases {:dev {:extra-paths ["dev"]
:extra-deps {org.clojure/tools.namespace {:mvn/version "0.2.11"}
org.clojure/tools.deps.alpha {:mvn/version "0.5.460"}
nrepl/nrepl {:mvn/version "0.4.5"}
reply {:mvn/version "0.4.2"}}
:main-opts ["-m" "nrepl.cmdline"]}}}
#2018-11-0316:52lwhortonmakefile:
NREPL_PORT=$(shell cat .nrepl-port)
dev:
clj -A:dev
.PHONY: dev
clj-repl:
clj -R:dev -m reply.main --attach $(NREPL_PORT)
#2018-11-0316:55lwhortonand my user.clj file (pulled in from the extra-paths in deps.edn):
(ns user
(:require
[ :as io]
[clojure.tools.namespace.repl :refer [refresh refresh-all]]
[clojure.pprint :refer [pprint]]
[clojure.inspector :as i]
[gerbil.core :as c]
)
)
(def ^:dynamic system nil)
(defn init []
(alter-var-root #'system (constantly (c/system))))
(defn start []
(alter-var-root #'system c/start))
(defn stop []
(alter-var-root #'system
(fn [s] (when s (c/stop s)))))
(defn go []
(init)
(start))
(defn reset []
(stop)
(refresh :after 'user/go))
#2018-11-0316:56lwhortonyou can drop the reply stuff if you want and just use nrepl’s “basic” client#2018-11-0311:26Christian JohansenI'll third that THANK YOU! :+1:#2018-11-0312:10slipset@lwhorton juxt pack generates jars for you #2018-11-0315:56orestis@borkdude :cider-nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.4.5"}
cider/cider-nrepl {:mvn/version "0.18.0"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}
:nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.4.5"}
org.clojure/tools.nrepl {:mvn/version "0.2.13"}}
:main-opts ["-m" "nrepl.cmdline"]}#2018-11-0315:56borkdudethanks#2018-11-0315:57orestisHere’s what I use to launch nREPL — the first variant is what I actually use to connect CIDER.#2018-11-0316:11cap10morganAsking this here too since it’s not really Datomic-specific: I have a datomic-pro download in a dir (so it has the datomic-pro-version.jar at the root, a lib subdir with deps jars, etc. I am trying to setup a tools-deps project to run some clojure code using datomic and all of its deps w/o triggering a download of them. Is there a more efficient way to do it than having to specify each individual jar in the :paths vector in deps.edn?#2018-11-0316:29cap10morganAnswering my own question:
:paths ["src" "/path/to/datomic/datomic-pro-version.jar" "/path/to/datomic/lib/*"] was the answer. (Not "/path/to/datomic/lib/" nor "/path/to/datomic/lib/*.jar")#2018-11-0506:55slipsetI see that https://github.com/healthfinch/depstar/ and https://github.com/juxt/pack.alpha both are in the business of packaging your project where depstar seems a bit more focussed on uberjars. Any thoughts on this?#2018-11-0506:57slipsetReason I’m asking is that I’m pondering creating a deps-shipit which would generate/refresh the pom, update the versions in it, create your choice of (skinny/uber) jar and then upload it to Clojars.#2018-11-0507:58dominicmUberjars aren't really suitable for uploading to clojars#2018-11-0508:07slipsettrue enough#2018-11-0508:07dominicm@slipset fwiw, I'd be interesting in a tool which only concerns itself with uploading jars to maven repos. Maybe with support for s3 wagons too.#2018-11-0508:09dominicmHaving a tool which starts a JVM to run this shell script isn't immediately on my list of problems I need solving:
#!/bin/sh
clojure -A:pack -m …
clojure -Spom
clojure -A:shipit output.jar pom.xml
#2018-11-0508:09slipsetoh, like https://github.com/slipset/deps-deploy ?#2018-11-0508:10dominicmThis is fantastic ❤️ 2 comments:
1. Can we look at getting gpg support for credentials, I think this is really important. Environment variables are far from secure.
2. I'd love to have this replace my current documentation in pack for using mvn:deploy-file 🙂#2018-11-0508:11slipset1) Yes. If I could ask you to file an issue/submit PR, that’d be fantastic#2018-11-0508:11dominicmI asked for this to exist about a week ago, so I'm glad it now does. I'm going to have to try imagining more things. Pile of Gold. Pile of Gold.#2018-11-0508:12slipset2) I’m more than happy to improve it in any way 🙂#2018-11-0508:12slipsetIn fact, I was about to submit a PR against pack suggesting the use of deps-deploy this morning 🙂#2018-11-0508:16slipset@dominicm the gpg support for credentials, that would basically be supporting passing private-key-file to aether, right? https://github.com/cemerick/pomegranate/blob/master/src/main/clojure/cemerick/pomegranate/aether.clj#L325#2018-11-0508:16dominicm@slipset I don't think so. I am not aware of that feature at all though 🙂#2018-11-0508:17slipsetOk, then if you could outline a solution in a ticket, I’d be very happy. I haven’t worked much with gpg.#2018-11-0508:18dominicmI've put quickly jotted down some thoughts, I've tried to capture my overarching goal of "moar secure" and provide some options.#2018-11-0508:19slipsetThank you! I’ll have a closer look this evening.#2018-11-0508:20slipsetBtw, concerning a potential deps-shipit, you’d actually end up just firing up one jvm, as the plan for deps-shipit would be to compose the three steps you are outlining into one.#2018-11-0508:23dominicmI know 🙂 But it's similar to this also:
#!/usr/bin/env clojure
(require '[pack.onejar])
(require '[deploy])
(require '[t.d.a])
(pack.onejar/jar)
(t.d.a/generate-pom)
(deploy/deploy jar pom)
Having said that, I'm usually happier to automate myself than others are 🙂#2018-11-0508:24slipset🙂#2018-11-0508:28dominicmSupporting jar signing seems important too, trying to find some good references on it#2018-11-0508:29slipsetShouldn’t be too hard, lein already provides this. Just need to RTFS a bit.#2018-11-0508:36dominicmyeah. Reading the source looks like the best option, given that maven + gpg signing isn't giving me anything useful.#2018-11-0512:21rickmoynihanIn my experience gpg and lein has been a bit awkward to setup… and I’ve seen lots of people struggle to do it. Is there an option to use something like bouncy castle here ( https://www.bouncycastle.org/java.html )? Or perhaps something that ships with the JVM?
I’m assuming the main thing gpg brings to the table (aside from the crypto) is being able to interact with the OS keychain.#2018-11-0512:21rickmoynihanNot sure if bouncy castle has any support for that#2018-11-0513:04dominicm@rickmoynihan You've highlighted my concern with BouncyCastle immediately. It would be good to enumerate the exact issues people have. I would expect it's around gpg vs gpg2. But I'm potentially projecting the issue I personally have.#2018-11-0513:04dominicmI would expect using Bouncy Castle with PGP brings in a new class of issues, like support smart cards.#2018-11-0513:05dominicmOne option is to search & present options. e.g. if only gpg/gpg2 are found, use those. If both, then ask the user.#2018-11-0513:35rickmoynihanYeah; I suspect gpg/gpg2 are the best options… as lein and others have been through this before and everyone always seem to settle on gpg. It’s just frustrating that that choice often leaves to brittleness and installation/configuration issues… but I guess it’s always going to be a bit of a cross platform nightmare 😞#2018-11-0513:46dominicmIt doesn't have to be I think 🙂#2018-11-0513:46dominicmLein also doesn't prompt the user ever. There are other mistakes there.#2018-11-0514:15rickmoynihanI can’t really recall my issues with gpg… it could well have been gpg vs gpg2. IIRC mine might also have been due to config cruft from .bashrc which I ported from linux to mac years back. In my experience it’s just one of those areas which can be fiddly, where you really don’t want something fiddly.#2018-11-0514:59dominicmIf gpg is hard to use, I'd be inclined to not tackle the problem 🙂#2018-11-0514:59dominicmBut that's just me.#2018-11-0514:59dominicmWhat I mean is, if gpg is easy to shoot yourself in the foot with, that's gpg's problem. I don't think it's a good use of time to handle the thousands of config combinations.#2018-11-0515:30slipsetlein actually prompts me for my gpg thing (if I don’t turn off the signed artifacts thingy)#2018-11-0515:43dominicm@slipset your gpg passphrase you mean?#2018-11-0516:00slipsetyes, I guess that’s the correct term.#2018-11-0516:04seancorfieldAfter some updates to software on my Mac I could never get gpg working again so I just told leiningen not to sign things. I mostly use boot to upload to clojars to avoid gpg anyway. #2018-11-0516:06seancorfieldAnd I have my clojars credentials in my dot file unencyrpted - again so I can avoid trying to deal with gpg. #2018-11-0516:06seancorfield(And I was never able to get anything to work on windows in this area)#2018-11-0516:08seancorfieldPhil was always the one pushing gpg. No one used to bother until he bugged everyone 🙃 and the boot crowd never pushed it. #2018-11-0516:15dominicm@seancorfield do you mean exclusively with lein it broke, or it's broken altogether?#2018-11-0516:15dominicm@slipset I was thinking more about prompting if both commands gpg and gpg2 are available. Things like that.#2018-11-0516:16slipset@dominicm I’ll look into https://github.com/slipset/deps-deploy/issues/2 later tonight.#2018-11-0516:16dominicmNo rush. I think it's good to think on & compare.#2018-11-0516:16dominicmHammock time 🙂#2018-11-0516:16slipsetI’d be really happy if you could comment on my comment there so I have something to hammock on 🙂#2018-11-0516:17dominicmSorry, I've got my emails turned off 😄#2018-11-0516:17slipsetNothing to be sorry about, it’s actually very smart if you want to get things done 🙂#2018-11-0516:17dominicmWe have a large number of (non-)errors in a production environment at the moment, only way I can stay sane#2018-11-0516:38rickmoynihanFWIW on mac with GPG this is what I use and it seems to work: https://gpgtools.org/#2018-11-0517:39seancorfieldYeah, that's what I was using but it stopped working and uninstalling/reinstalling didn't fix it so I gave up. The whole thing seemed more hassle than it was worth -- given the only thing that seemed to care about it was Leiningen...#2018-11-0518:47dominicmI consider gpg an important part of my security system, I think its a shame more people don't use it with jars.
But I can't speak to how well the gnu toolchain works elsewhere. #2018-11-0519:09slipset@dominicm https://github.com/slipset/deps-deploy/pull/4 if you could have a look?#2018-11-0519:10slipsetI used gpg --encrypt .clojars_creds.edn to encrypt my creds.#2018-11-0519:10slipsetwhich may or may not be what you were asking for, as this is a symmetric encryption.#2018-11-0519:13dominicmI'll see if I can get it working with my smart card. Looking at the code, I don't expect it will, as I don't have a gpg passphrase.#2018-11-0520:34slipsetMay I ask what kind of smart card you have?#2018-11-0521:16dominicmYubikey#2018-11-0616:42timgilbertSay, are there options available for customizing the groupId, version, etc in pom.xml during the course of clojure -Spom?#2018-11-0616:42dominicm@timgilbert update the generated pom.xml, then -Spom won't override them.#2018-11-0616:43timgilbertHrm. I don't love having half-generated code in my codebase that I'll need to create another git commit for whenever I update deps.edn#2018-11-0616:46timgilbertI suppose I could write a tool that would run -Spom and then update the relevant values. What I'm actually looking for is something that will create the version number based on git tags on my repo in the same way lein-v does#2018-11-0617:01dominicmI imagine you're into "write another tool" territory 🙂#2018-11-0617:04arrdemIf someone writes a manifest generator, let me know 😉 I’m on the hunt for one.#2018-11-0617:21seancorfield@arrdem I forked depstar and added basic manifest generation -- then decided I really didn't need it and removed it.#2018-11-0617:21ghadiheh#2018-11-0617:23arrdemyagni#2018-11-0617:24arrdemI’m somewhat surprised how far bare jars have gotten me so far - we’ll see.#2018-11-0617:25ghadiI'm surprised by all the git deps users wanting to have artifacts back#2018-11-0617:25seancorfieldWe build/deploy bare uber JARs and run them that way in production. No AOT. java -cp path/to/the.jar clojure.main -m entry.point#2018-11-0617:34arrdemAt this point I’m convinced that between Katamari / git deps I’ll probably only use external libraries from Maven again. Which is great. But I have internal users who aren’t on that workflow yet and who I want to be able to deploy artifacts with manifests and poms for.#2018-11-0617:35ghadiI get it from a transition strategy or reach perspective#2018-11-0617:35arrdemmaking nicer / faster psuedo-uberjars in docker is easy enough, doesn’t require any of this and that’s my only deployment need.#2018-11-0619:02orestisCan someone share some knowledge about the difference between bare jars and Uberjars and what manifests are for?#2018-11-0619:02orestis(Links welcome, no need to type it out here)#2018-11-0619:02orestisI finally got that Pom.xml is a maven thing :)#2018-11-0619:07orestisI’d love to be able to just rsync individual jars instead of an Uber jar, then somehow launch my app with the correct classpath. I guess if you have clj command line on the server installed, you can do that but you pay for the startup compilation time. #2018-11-0619:08dominicm@orestis Check out pack's skinny jar mode 🙂#2018-11-0619:15seancorfield@orestis FYI, we deploy source uberjars to production -- no AOT -- and with all their dependencies included, they're still only 20-30MB so it's a pretty small payload to upload across servers (but, yes, would take a while to upload from a dev machine I guess).#2018-11-0619:21orestisI tried AOT jars and they still are roughly 30-40MB so still pretty small. It’s the compiling that takes time :)#2018-11-0619:23orestisAnd it’s annoying that you compile the same dependencies over and over. I’m fairly sure I’ve seen Alex & co discussing this as a potential improvement in the future. #2018-11-0710:38rickmoynihanYes, I saw this the other day on the clojure mailinglist and was excited by it too.
It’s a mild annoyance that uberjaring and AOTing a moderately sized app takes 2 minutes. I understand why @U04V70XH6 likes to avoid AOTing apps; I’m assuming to have quicker builds… and I’ve been contemplating only AOTing nightlies, to bring CI times down… but still have uberjars start quickly.
I think this problem could be solved if we could easily AOT libraries. I can’t recall what the issues are here; I suspect it’s to do with how clojure.core/compilation/read/eval get linked in to libraries.#2018-11-0715:33seancorfieldOne big problem with AOT for libraries is that they can locked into specific versions of Clojure (this was a problem back in the day when some did compile their libraries for distribution). #2018-11-0715:33seancorfieldWith source-only libraries they should always work with all future versions of Clojure. #2018-11-0715:37seancorfieldAOTing an application is safe because it's an entire system. Self-contained. For us, the startup time hit just isn't worth adding an AOT step. New Relic's instrumentation at startup is a much bigger overhead for us than Clojure's compilation. #2018-11-0716:35orestisOh, can you share that new relic thing please?#2018-11-0716:45seancorfield@orestis What do you mean? We just use the regular Java agent for New Relic. I think we had to blacklist a particular classloader in Clojure to avoid prohibitively slow NR startup, but otherwise you just add the JVM startup options to point at their library and config file.#2018-11-0716:46seancorfieldhttp://corfield.org/blog/2016/07/29/clojure-new-relic-slow-startup/ -- talks about the slow startup issue.#2018-11-0716:46seancorfieldhttp://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/ -- this covers how to add Trace annotations to Clojure functions for use with New Relic.#2018-11-0716:47seancorfieldHappy to answer any other specific Qs you might have.#2018-11-0716:50orestisRight, that! Thanks so much!#2018-11-0703:25seancorfieldDeleted the last vestiges of Boot from our system today (on dev). I decided to even convert two legacy system pieces since they are going to be around for a few months and it seemed silly to keep Boot around just for those. Very pleased with nice, simple, lean processes we have now. And we're on Clojure 1.10 Beta 5 in dev as well. Several blog posts to come 😉#2018-11-1004:18jwhitlarkThose blog posts are eagerly awaited. Do you think tools.deps has removed the need for boot, in general?#2018-11-1005:16seancorfieldI think it depends what you want from a build tool. #2018-11-0707:57cflemingI’ve talked about some variant of this in the past, but I can’t remember the exact details now. It seems to be impossible to use tools.deps as a library from a process which doesn’t have the CWD set to the directory containing the main deps.edn file. This seems like a significant limitation - it means you can never have a process which resolves deps from two separate deps.edn files, for example.#2018-11-0707:58cflemingThe problem is that if your deps.edn file references another using :local/root, the paths are incorrect - the paths from the dependency are relative to the CWD, not to the deps.edn file location.#2018-11-0708:01cflemingThe problem is here: https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions/deps.clj#L29#2018-11-0708:02cflemingHere, root is the local :deps/root, which is generally relative.#2018-11-0708:02cflemingIs this intentional? If not, I’ll file a bug.#2018-11-0708:06cflemingI was somewhat convinced that this wasn’t previously the case, but it seems to have been this way for some time according to git.#2018-11-0708:07dominicmAlways been this way, yeah#2018-11-0708:13cfleming@dominicm Is there an issue for it? There’s TDEPS-74 but that’s the transitive one. I can’t see anything else likely-looking.#2018-11-0708:22mpenetI think that also might be one of the reason katamari has vendored/forked deps for its use#2018-11-0708:24cflemingInteresting, I’ll have to check that out.#2018-11-0708:28cflemingRight, looks like it uses raynes/fs#2018-11-0708:30cflemingThis is kind of a show-stopper for Cursive, unfortunately.#2018-11-0708:31dominicmUncertain. I think there should be. #2018-11-0717:08orestisWhat about CLJ_CONFIG? I remember @seancorfield uses it for something similar?#2018-11-0717:18dominicm@orestis that's for having multiple deps.edn files. It's bad for tooling as it stops users from adding personal aliases to use.#2018-11-0717:24seancorfieldWell, it's good for projects that want a specific deps.edn "base" file instead of using whatever the developer happens to have in their .clojure/deps.edn file -- so it really depends on your perspective.#2018-11-0717:27seancorfieldFor our monorepo, we specify all our defaults (some aliases, but mostly :override-deps) in a versions/deps.edn file and we use CLJ_CONFIG to use that as our basis for all the subprojects' deps.edn files.#2018-11-0717:28seancorfieldThat doesn't help Cursive tho', since it affects the "wrong" deps.edn file -- the "user" one, not the "project" one.#2018-11-0717:29seancorfieldIn one place, we shell out to the clojure command and when we do that (via clojure.java.shell/sh) we specify the :dir that it should be executed in so CWD is controlled. I'm not sure how Cursive invokes this stuff -- I suspect it is trying to use t.d.a. directly and that's where the problem occurs...?#2018-11-0717:33dominicmcursive is java, so it's using it as a library rather than shelling out. I think @cfleming mentioned that $PATH isn't set in cursive.#2018-11-0717:38seancorfieldIn our Clojure code, we still shell out to run clojure 🙂#2018-11-0718:53cflemingWhat I want is to get the full Java command line that I need to execute, so that I can execute it with IntelliJ’s command running infrastructure. There’s no way to get that from the command line with clj.#2018-11-0718:58cfleming@dominicm The issue is that if you use t.d.a as a lib, there’s no way to set the CWD of a running Java process. It’s not a $PATH thing.#2018-11-0719:36mpenetApparently there's a way with jruby https://github.com/arohner/lein-daemon/blob/master/daemon-runtime/src/leiningen/daemon/runtime.clj#2018-11-0719:36mpenetI guess it's jni under the hood#2018-11-0720:29cflemingYeah, but that would probably break all sorts of things with IntelliJ (where the logs go, etc).#2018-11-0719:19dominicmDuh, of course #2018-11-0719:32rickmoynihan@cfleming: is an entirely insufficient workaround to launch idea from the command line in your projects working directory?#2018-11-0719:34rickmoynihanIIRC I’ve done that in the past with intellij; when I’ve run into issues with macos/quartz-UI not being a child of a shell environment#2018-11-0719:35rickmoynihanthough I don’t normally use intellij#2018-11-0720:10arrdem@rickmoynihan yeah - most IDEA launches are through a GUI, and that defeats IDEA’s multi-project/workspace capabilities.#2018-11-0720:13rickmoynihan… but it does work around the issue?#2018-11-0720:27cfleming@rickmoynihan Only if you ever use one project - I usually have 3 open.#2018-11-0720:30rickmoynihanas I said an entirely insufficient workaround#2018-11-0720:28cflemingI actually don’t even know how to run a Mac app from a specified directory, or if it’s even possible.#2018-11-0720:30rickmoynihan/Applications/IntelliJ\ does it for me#2018-11-0720:30cflemingFor now, I’ll probably just have to do the same horrible hacks I do for leiningen using hooke or the moral equivalent.#2018-11-0720:32cfleming@arrdem Has raynes/fs worked well for your vendored fork?#2018-11-0720:32cflemingLooking at the code, I assume this sort of problem is what you’re trying to deal with?#2018-11-0720:32arrdem@cfleming Yeah. It was a bit fiddly to figure out where I needed it, but I haven’t had problems with it since.#2018-11-0720:33arrdemYou’re right, that is why I patched it in.#2018-11-0720:33cflemingAre there places where you don’t want it? If I use hooke, then that’s pretty global.#2018-11-0720:34arrdemCan you hooke or alter-var-root on io/file? That seems like a really big hammer and may play badly with static linking.#2018-11-0720:35arrdemIn general I couldn’t prove that things wouldn’t behave precisely as expected and erred on the side of caution, so I did the refactor by hand and it took a couple passes.#2018-11-0721:47cfleming(sorry, got called away there)#2018-11-0721:47cflemingYeah, that’s what I do. It’s scoped to a particular sequence of calls, but during that sequence it’s globally modified, yes.#2018-11-0721:48cflemingIt seems to work, and I think it works because the code calling io/file is not statically linked, so it uses the var.#2018-11-0721:48cfleming@arrdem ^^#2018-11-0721:49arrdemhummm#2018-11-0721:49arrdemexciting#2018-11-0721:49cflemingI’m not sure whether that’s better or worse than vendoring a specific version of deps, which is then tied to that version of Cursive.#2018-11-0721:50arrdemFor katamari it’s as much a proof that katamari’s heavily vendored monorepo workflow actually does work.#2018-11-0721:50cflemingAll options suck, basically.#2018-11-0721:50arrdemfork && vendor && move on#2018-11-0721:51cflemingI can’t wait for the bug reports saying that Cursive xxx doesn’t support the functionality that was introduced in deps yyy#2018-11-0721:52arrdemyeah I need to write some bash + sed automation to let me packport merged diffs ~automatically.#2018-11-0722:12cflemingThe problem I have with that is that I can’t rely on everyone upgrading Cursive, so older versions will still be out there with older versions of tools.deps baked in.#2018-11-0813:21onetombtw, we are relying more and more on project specific nix-shells (activated via [`direnv`](https://direnv.net) by simply cding into a project's directory)
and intellij doesn't expects such approach at all, instead it's providing tons of custom dialogs with very similar path and option settings for the various tools in different plugins 😞
i really envy emacs in this respect, since they have https://github.com/wbolster/emacs-direnv#2018-11-0815:56richiardiandreaDefinitely nice will try it out#2018-11-0813:59dominicmNeat, it supports vim too 🙂#2018-11-1005:57jwhitlarkPSA, there's work going on with docker-clojure: https://github.com/Quantisan/docker-clojure/pull/56#2018-11-1107:43jwhitlarkI was playing around with https://clojure.org/reference/compilation#_gen_class_examples and I've noticed that with 1.10.0-beta5, I couldn't compile the example code until I'd created a classes directory. Is that expected behavior?#2018-11-1107:50jwhitlark#2018-11-1108:18dominicmIt is normal behaviour from my past testing on this. The output directory is supposed to already be in the classpath. #2018-11-1108:31jwhitlarkOk.#2018-11-1108:53orestisDoes tools-deps have a story for compiling Java code that lives in the same repo?#2018-11-1112:43danierouxJoined to ask this exact question. I have one measly .java file that I want to (:import), and i really don't want to make a jar just for that one file.#2018-11-1108:53orestis(No immediate use case, just curious)#2018-11-1109:02jwhitlarkI'm curious about that too.#2018-11-1109:24dominicmNot at all @orestis. #2018-11-1109:25dominicmI've done some work on this, but it would be the same level as clojure.core/compile I suspect. Riddler is a good option to look at I think#2018-11-1112:46danierouxwhat is Riddler, where can I find it?#2018-11-1113:13dominicmApparently I got mixed up, I meant this: https://github.com/ztellman/virgil#2018-11-1113:02dottedmagtools.deps is about building classpath, clj is about running Clojure. You can run javac from one of clj aliases if that's a single .java file.#2018-11-1123:17jwhitlarkWow, digging through virgil's code answers a lot of questions.#2018-11-1200:54gfredericksthere's no way to add a jvm-opt without having to enable an alias via CLI opts, is that right?#2018-11-1201:34mfikes@gfredericks -J#2018-11-1201:35gfredericks@mfikes sorry, I mean I'd like to put it in the deps.edn#2018-11-1201:35mfikesExample: clj -J-Xmx512m#2018-11-1201:35gfredericksand have, e.g., clojure pick it up#2018-11-1201:35gfrederickswithout any extra args#2018-11-1201:36mfikesAhh, I think you are right in that case. I can't think of a way to just "make it so"#2018-11-1201:38mfikesIt would seem, a top-level thing, sibling of :deps or :paths is where a :jvm-opts could live#2018-11-1201:39gfredericksyeah; I can't tell if that is just a missing feature, or if there's some philosophical conflict#2018-11-1201:39alexmillerIt’s missing, no philosophical objection#2018-11-1219:11adamfreyhere's a command that I can run from the terminal to start a cider-ready nrepl, but I'm having a problem when putting it into an aliase in a deps.edn file.
clj -Sdeps '{:deps {nrepl {:mvn/version "0.4.5"} cider/cider-nrepl {:mvn/version "0.18.0"} refactor-nrepl {:mvn/version "2.4.0"}}}' -m nrepl.cmdline --middleware '[cider.nrepl/cider-middleware refactor-nrepl.middleware/wrap-refactor]'
the --middleware option is a vector with a space in the middle, but I can't find any way to make it work from deps.edn.
:main-opts ["-m" "nrepl.cmdline"
"--middleware" "[cider.nrepl/cider-middleware refactor-nrepl.middleware/wrap-refactor]"
"--interactive"]
by the time it gets to the -main function the args binding is set to
("--middleware" "[cider.nrepl/cider-middleware" "refactor-nrepl.middleware/wrap-refactor]" "--interactive")
#2018-11-1219:36dottedmag@adamfrey try , instead of a space#2018-11-1219:39adamfreyoh good call! it works because it gets passed as one thing to the nrepl main and then nrepl parses it from a string using edn/read-string Thanks!#2018-11-1221:53martinklepschAlso known as the Corfield Comma 😂#2018-11-1219:56aviI seem to be bumping up against a dep conflict but I’m not quite understanding the behavior I’m seeing from tools.deps + CLI-tools; since I had a lot of text to share I started a topic over at ClojureVerse. If anyone has a minute to take a quick look and possibly enlighten me as to what I’m missing, I’d much appreciate it!
https://clojureverse.org/t/tools-deps-not-downloading-all-deps-when-run-with-r-aliases/3146?u=avi#2018-11-1220:22seancorfieldAnswered on ClojureVerse.#2018-11-1220:22aviThanks Sean! 🙏#2018-11-1220:23seancorfieldTL:DR; dependency version resolution has a different set of versions to consider with one alias vs multiple aliases.#2018-11-1220:24aviI think I get it; makes sense 😅#2018-11-1516:01aviJust wondering if anyone’s figured out a way to get tools.deps to use a http proxy when downloading Maven deps, other than applying the patch in https://dev.clojure.org/jira/browse/TDEPS-20 ? I’m just being lazy I guess, putting off learning how to patch tools.deps and install and use the patched version…#2018-11-1516:07alexmillerI don’t think there is any way to do it but to fix that (and would be happy to see a patch for it)#2018-11-1516:07alexmilleroh, there is a patch for it, sorry#2018-11-1516:07alexmillerwell, then you just have to wait :)#2018-11-1516:08alexmillerI’m probably not going to have much time to look at tools.deps stuff until post-conj#2018-11-1516:54avitotally understandable, no worries! Thanks for the clear and helpful response!#2018-11-1517:34martinklepschIs there any documentation on how tools.deps resolves deps? Specifically wondering what happens when two dependencies depend on 'A but in different versions#2018-11-1517:35alexmillerthe general idea is to always take the newest#2018-11-1517:36martinklepschOk, that's great. Does that "general idea" imply any exceptions to that rule where the older artifact would be chosen?#2018-11-1517:36alexmillerthe complexities come when you consider the full graph#2018-11-1517:37alexmillerA -> B1 -> C2
-> B2 -> C1#2018-11-1517:37alexmillerdo you get {A, B1, C2} or {A, B2, C1} or {A, B2, C2}#2018-11-1517:38timgilbertI'm glad I'm not the one who needs to figure that out 😉#2018-11-1517:39alexmillerit’s actually NP-complete I believe#2018-11-1518:10hiredmanhttps://research.swtch.com/version-sat#2018-11-1518:14hiredmanhttps://cwiki.apache.org/confluence/display/MAVENOLD/SAT+Based+Dependency+Resolution#2018-11-1517:39alexmillerin the general case#2018-11-1517:42alexmillerfortunately most actual cases are pretty “easy”#2018-11-1517:52timgilbert...unless they involve jackson, I'm guessing#2018-11-1518:00alexmilleryes, that is the exception :)#2018-11-1518:00alexmillerand unfortunately, every dependency set includes at least 9 versions of jackson#2018-11-1518:56andy.fingerhutA new complexity class: jackson-complete#2018-11-1518:56andy.fingerhutAlthough that might be equivalent to: undecidable 🙂#2018-11-1518:58mfikesEntjaksonsproblem#2018-11-1519:06andy.fingerhutOh, that is good. You even Deutsch-ified the spelling of Jackson. Love the attention to detail there 🙂#2018-11-1519:13alexmillernow that’s a t-shirt with sufficiently small audience#2018-11-1519:16andy.fingerhutA sufficiently small audience to .... not make it?#2018-11-1519:25alexmillerto be fun :)#2018-11-1519:27alexmillera joke t-shirt with an audience of 1 is not fun, because you can’t share it. And a joke t-shirt with an audience of large N, is cliche. So you have to find that sweet spot in the middle. :)#2018-11-1519:27alexmillerthis is bigger than 1, but not by much :)#2018-11-1519:29andy.fingerhutGot it. It is pretty easily explained to anyone at the Conj, if they don't get it immediately (I wouldn't without this conversation)#2018-11-1519:32andy.fingerhutAs a reminder, if you think Rich would see a "channeling Rich sayings" T-shirt as a tribute (as it is intended), not a weirdness or offensive, I would be happy to bring some to the Conj 🙂#2018-11-1519:39alexmillerprob not the best idea#2018-11-1519:39andy.fingerhutBut also happy to never bring it up again if he doesn't want to see such things.#2018-11-1519:39alexmillerfor someone with such great care for words, I hate to ascribe any words to him#2018-11-1519:41andy.fingerhutUnderstood. He isn't the only person like that, and such people should get to be upset with people that misquote them, especially if the meaning is changed. FYI, I do take pretty significant care in transcribing his talks. I think I've listened to the ones I have transcribed at least 4 times each now.#2018-11-1519:41seancorfieldMaybe we should make some (rich (comment forms))
T shirts? off-topic#2018-11-1519:42alexmillerI don’t get that#2018-11-1519:42alexmillerI appreciate the care in the transcripts - he’s never complained about those#2018-11-1519:42seancorfieldStu talked about Rich Comment Forms in his "Running With Scissors" talk.#2018-11-1519:42alexmillerand I know he appreciates having slides+text together for context#2018-11-1519:42alexmilleroh, I haven’t watched it yet :)#2018-11-1519:43seancorfieldAnd observing it was a pun on the (comment ...) forms being "rich" with exploratory code and it also being a habit of Rich to use comment that way.#2018-11-1519:44alexmillerclearly the circle is too small for a t-shirt :)#2018-11-1519:44alexmillerfwiw, I made a custom “O(log32 n) is good enough” t-shirt for Conj 1 :)#2018-11-1519:54dpsuttonif you donated that design to clojurist together i bet they could raise some money for projects by selling the t shirt. i would for sure get one#2018-11-1520:06alexmillerwell feel free to make one yourself, mine was certainly nothing special and I don’t have it#2018-11-1520:09andy.fingerhuthttp://customink.com makes it really easy to create T-shirt designs, I found out recently#2018-11-1520:10andy.fingerhutI have never bought anything from them, so don't have knowledge of the full experience front to back.#2018-11-1520:13dpsuttoni made some tshirts for my team at my last company through them. I still get emails to make mouse pads and other things with that design 2.5 years later#2018-11-1520:13dpsuttonbut the shirts were/are great#2018-11-1601:28gfredericks...mouse...pads?#2018-11-1601:28gfredericksI don't think I've even seen a mause pad for at least fifteen years#2018-11-1519:45andy.fingerhutI guess we'll have to live with these continuing to be the only Rich-approved fan club T-shirts: https://clojure.org/community/swag#2018-11-1519:46alexmillerwell, all the conference t-shirts too :)#2018-11-1520:18cflemingI still love my conj 2014 t-shirt.#2018-11-1520:18dpsuttonmy clojure west 2016 and strangeloop 2016 are my two favorite shirts#2018-11-1520:18cflemingNot many other conference t-shirts stick around for long.#2018-11-1520:19andy.fingerhutAnd here I thought the biggest benefit of the conferences was listening to the talks, and the hallway track 🙂#2018-11-1520:20cflemingNo, listening to the talks is totally overrated, it’s only the hallway track 🙂#2018-11-1520:20cflemingAt least for me, since I can watch talks online, but seldom get to chat with others who understand what I’m talking about.#2018-11-1520:21andy.fingerhutBut clearly I have missed out on some great T-shirt buying opportunities that are not available elsewhere. This should be in the conference advertisements!#2018-11-1609:05martinklepschhas there been any discussion around preventing things like this:
(tdeps/resolve-deps
{:deps '{clj-time {:mvn/version "0.8.0"}
clj-time/clj-time {:mvn/version "0.7.0"}}
:mvn/repos default-repos}
nil)
#2018-11-1613:14alexmillerShould be fine - there is a canonicalization step#2018-11-1617:51tcrawleyI'm working on migrating a project from lein to tools.deps, and need to access an authenticated repo. https://clojure.org/reference/deps_and_cli#_procurers says that the credentials are pulled from ~/.m2/settings.xml, which doesn't work well for us. Is there another method for specifying credentials, or plans to expose one?#2018-11-1617:51alexmillerin short, no#2018-11-1617:52tcrawleyI'd be fine with embedding the credentials in deps.edn - ours are generated by make, and are never checked in#2018-11-1617:52alexmillerwe are not fine with putting any kind of credentials in deps.edn so will not be doing that#2018-11-1617:53alexmillerperhaps there is some other way to pull credentials from a trusted source#2018-11-1617:55tcrawleyWhat about a tools.deps-specific file for them? ~/.tools-deps.edn would possibly work, and is as secure as ~/.m2/settings.xml#2018-11-1617:56dominicm@tcrawley settings.xml is more secure. It supports encryption. Unless tools.deps adds encryption.#2018-11-1617:56alexmillerwhy is that different for you from making a ~/.m2/settings.xml?#2018-11-1617:56alexmillerhave no desire to do the encryption stuff#2018-11-1617:56tcrawley@dominicm does tools.deps support encrypted ~/.m2/settings.xml?#2018-11-1617:57alexmillerit uses the Maven apis, so should#2018-11-1617:57tcrawleyah, that makes sense#2018-11-1617:57alexmillercan’t say that I’ve actually done it#2018-11-1618:00dominicm@tcrawley if you wanted to get really crazy. You could generate a server into ~/.m2/settings.xml and use a reference to that in deps.edn.#2018-11-1618:00dominicmxml is just data after all.#2018-11-1618:01tcrawleyright, I'd just like to avoid that if possible :) I can output an alternate settings.xml, then tell tools.deps to use that if the maven libs support setting that via an environment variable and it actually works#2018-11-1618:01tcrawleyI'll try that first#2018-11-1618:05dpsuttonthere's also a bug with the credentials in settings.xml. its on github issues for s3 wagon maybe? alex had very useful information in it#2018-11-1618:05dpsuttonit affects my ubuntu coworkers but not me on fedora right now#2018-11-1618:05dpsuttonhttps://github.com/s3-wagon-private/s3-wagon-private/issues/31#2018-11-1618:18alexmillerthat is not an issue with tools.deps (as I am using a different version than s3-wagon-private)#2018-11-1618:19alexmillerthe stuff documented at https://clojure.org/reference/deps_and_cli should work#2018-11-1618:29kennyWhy does this produce a REPL instead of running my main opts?
clj -Sdeps '{:aliases {:jar {:extra-deps #:com.healthfinch{depstar {:git/url "", :sha "2879672df7fd296855563c22dd9ba9de2022a0b4"}}}, :main-opts ["-m" "hf.depstar.uberjar" "foo.jar"]}}' -A:jar
Clojure 1.9.0
user=>
#2018-11-1618:29ghadibecause the ns'ed map syntax is qualifying :sha incorrectly#2018-11-1618:29kennyWow, :main-opts is at the wrong level. Oops...#2018-11-1618:30kennyGreat start to the day 🙃#2018-11-1618:30ghadimain opts isn't supported top-level, AFAIK#2018-11-1618:30ghadibut yeah it's at the wrong level#2018-11-1618:31kennyShouldn't there be some validation there?#2018-11-1618:31kennyI passed a vector instead of a map to an alias.#2018-11-1621:28tcrawleyI figured out how to tell the maven lib to use a different settings.xml file by setting the org.apache.maven.user-settings sysprop. However, clojure doesn't expose a way to pass options to the make-classpath java invocation. Is that something you would be interested in exposing @alexmiller?#2018-11-1621:29tcrawleyI can work around it with a custom copy of the clojure script for now#2018-11-1621:31alexmillerWould prob rather add a top level :mvn/ key but would be good to have a ticket regardless#2018-11-1621:32tcrawleyI was thinking about a general way to pass options to that invocation, but a top-level :mvn/settings-file (or whatnot) would work for this use-case. I'll file a ticket.#2018-11-1621:40dominicmI initially thought it would not work, but as long as -Sdeps is considered, it works quite well. #2018-11-1621:43tcrawley@dominicm what "it" are you referring to?#2018-11-1621:44dominicmA top level key rather than an environment variable @tcrawley #2018-11-1621:44tcrawleygotcha#2018-11-1718:25richiardiandreaLittle question, is there a way to override the information that tools.deps.pom/gen-pom uses below
https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/gen/pom.clj#2018-11-1718:26richiardiandreaI am trying to generate a pom with version and project coords coming from the outside#2018-11-1718:26alexmillerNo, but it syncs - so you can change it and it won’t be removed#2018-11-1718:28richiardiandreaYep I saw the sync but in my case the info is coming from another file (either project.clj or package.json) and I think it would be awesome to have the pom.xml generated somehow - not persisted. Is it something worth discussing about?#2018-11-1718:32richiardiandreaThis is the only missing piece for an automated - deps.edn + project descriptor - no manual steps flow 😃
I know there exists probably a maven plugin for bumping versions in a pom but ...it is no very lean#2018-11-1718:33richiardiandreaAnd thanks for the quick answer as well #2018-11-1718:37richiardiandreaOr I can write some bash but when this happens it always feels there is something hacky I don't know why simple_smile #2018-11-1800:20ro6as tools.deps stands today, if I'm co-developing an app+library locally, do I need to cycle my clj repl to refresh changes from the library?#2018-11-1800:40alexmillerDepends but you can probably just reload the namespace#2018-11-1806:21dominicm@robert.mather.rmm tools.namespace refresh will reload namespaces in local libraries. #2018-11-1919:06martinklepschThis fails to find org.jboss.naming:jnpserver:jar:5.0.3.GA. org.immutant/messaging has the jboss repo specified in it's pom but only transitively depends on jnpserver. The the immediate dependencies of org.immutant/messaging do not specify the jboss repository.
clj -Sdeps '{:deps {org.immutant/messaging {:mvn/version "2.1.10"}}}' -Stree
#2018-11-1919:06martinklepschIs this something worth opening a JIRA ticket for?#2018-11-1919:10alexmillerI think there is one?#2018-11-1919:12martinklepschYou're right: https://dev.clojure.org/jira/browse/TDEPS-46#2018-11-1919:12alexmillerWell that’s for -Spom right? Related but different#2018-11-1919:13alexmillerWell no, guess not#2018-11-1919:13alexmillerI find the way that ticket is written confusing#2018-11-1919:13martinklepschyeah... but there's a ticket for -Spom as well#2018-11-1919:27martinklepsch@alexmiller btw, I checked up on canonicalize-deps and either it's not working or I'm misunderstanding what it's supposed to do:
(let [cfg {:deps '{clj-time {:mvn/version "0.8.0"}
clj-time/clj-time {:mvn/version "0.7.0"}}
:mvn/repos mvn/standard-repos}]
(canonicalize-deps (:deps cfg) cfg))
;; => [[clj-time #:mvn{:version "0.8.0"}]
;; [clj-time/clj-time #:mvn{:version "0.7.0"}]]
#2018-11-1919:35alexmillerthere’s actually two different points where things get canonicalized#2018-11-1919:36alexmillerthe other is one of the extension methods that gets called by resolve-deps, and I think that’s where this gets adjusted#2018-11-1919:37alexmillerI started doing this earlier in canonicalize-deps and then ended up incorporating it later in the process instead#2018-11-1919:37alexmillersorry, don’t have time to look at it now#2018-11-1920:49borkdudeIn the case of git deps, which paths of the repo get added to the classpath? Does it look at a pom file? Can you control this? E.g. also add the test dir so you can run the tests of a git dep?#2018-11-1921:43alexmillergit deps will detect the manifest type (deps.edn or pom.xml) and use the paths indicated#2018-11-1921:46alexmilleryou can override it with :deps/manifest :deps or :deps/manifest :pom#2018-11-1921:48borkdudeI guess that’s only useful if both are present, else it will choose whichever is present?#2018-11-1921:48alexmillerprefers deps.edn#2018-11-1921:48alexmillerthen pom#2018-11-1921:48alexmillerif neither will throw, but if you explicitly set to :deps, it is tolerant of a missing deps.edn#2018-11-1921:48alexmillerand just treats it as a dep with no transitive deps#2018-11-1921:49borkdudein that case it chooses ["src"] as the default :paths?#2018-11-2208:30mkvlrI’m trying to get add-lib working with prepl. I’m using the latest add-lib branch and clojure:#2018-11-2208:30mkvlrclj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-beta7"} org.clojure/tools.deps.alpha {:git/url "" :sha "f6c080bd0049211021ea59e516d1785b08302515"}}}' -J-Dclojure.server.repl="{:port 5556,:accept clojure.core.server/io-prepl}" #2018-11-2208:32mkvlrto get it working I have to set the dynamic class loader first:#2018-11-2208:32mkvlr(let [cl (.getContextClassLoader (Thread/currentThread))
dcl (clojure.lang.DynamicClassLoader. cl)]
(.setContextClassLoader (Thread/currentThread) dcl))
#2018-11-2208:33mkvlris that how it’s supposed to work or wouldn’t it be nice if prepl did this for me like repl does? https://github.com/clojure/clojure/blob/b182982007df934394f0bc68b3a238ca9f200dd1/src/clj/clojure/main.clj#L389-L390#2018-11-2208:40mkvlror is adding functionality like this the job of the tool using prepl?#2018-11-2210:19Ben HammondHi guys I'm working on a project that extends
https://github.com/juxt/edge
which daisy-chains a bunch of small dep.edn files together using
{:local/root "../edge.kick"}
I'm trying to get my head round how this plays with alias declarations
⚓#2018-11-2210:20Ben Hammondso I've just discovered that it doesn't
aliases are only sampled at the root level#2018-11-2210:22Ben Hammondwhich implies to me that I have misunderstood the intention of :local/root#2018-11-2210:22Ben Hammondis there a description of what :local/root should, and should not be used for#2018-11-2210:22Ben Hammondthat you can point me at#2018-11-2210:23Ben HammondI guess I'm trying to strong-arm deps into working like an OSGI style module system#2018-11-2210:24Ben Hammondand I'm wondering if that would be considered abusive#2018-11-2210:39rickmoynihanI’ve had similar ideas about module systems.
In particular I have a nagging feeling that integrant/duct + deps.edn could be such a thing.#2018-11-2210:44Ben Hammondyes you are probably right#2018-11-2210:44Ben HammondI think my confusion is this#2018-11-2210:45Ben Hammond:local/root dependency allows you to see the internal development gubbbins of a dependency#2018-11-2210:46Ben Hammondwhich creates the temptation that you can manipulate those other aliases#2018-11-2210:47Ben Hammondbut at some point you should expect to move that dependence to a fixed binary#2018-11-2210:48Ben Hammondtherefore the thought process should always be How would I use this if it was a binary artefact#2018-11-2216:40mfikesThe fact that :local/root behaves just like :git/url or :mvn/version with respect to aliases is a good thing#2018-11-2312:39preporHello. I want to suggest another way how to build uberjars with tools.deps. boot-tools-deps amends boot's configurations. Our approach doesn't complement uber task but replaces it.
https://gist.github.com/prepor/ad40e061287793660c9faff21de23b82
This task simply adds classpaths (and paths) from tools.deps to fileset. And it works with git and local dependencies too. The final task could look like this:
(deftask build
[]
(comp (deps-uber)
(aot :namespace '[app.core])
(jar :file "standalone.jar"
:main 'app.core)
(sift :include #{#"standalone.jar"})
(target)))
#2018-11-2319:00dominicm@alexmiller https://dev.clojure.org/jira/browse/TDEPS-17 has become a priority for me. I can't use our git fork because it's coming in transitively.
I'd like to work on it this weekend, any thoughts you've formed so far? I'm thinking that the simplest version is that something like this will work:
org.clojure/tools.namespace {:mvn/version "1.0.0" :lib pro.juxt/tools.namespace}
Version is first priority. 2 with equal versions and different libs throws an error like comparing git to mvn does.#2018-11-2402:13eoliphanthi @alexmiller, running into an issue I posted over in #clojure
https://clojurians.slack.com/archives/C03S1KBA2/p1543000601943800
Just trying to understand if this is expected behavior vs a bug, etc#2018-11-2421:04jwhitlarkI haven't seen a bash/zsh completion script for clj, but it occurs to me that a) it wouldn't be that hard, b) the project specific part could use direnv to set that portion when you entered a clojure project directory. Any input?#2018-11-2510:26dominicmI have an interesting conundrum in pack. I'm trying to suppress this output:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
Suppressing it for my projects is easy enough, I add exclusions to pack for slf4j-nop.
However, on projects which don't already have their own logging stack, it then results in:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Which isn't a better result imo.
The ideal is that the -main function's in pack would tell slf4j what to use if there's multiple bindings, suppressing the warning. But I don't see an obvious API for that. Anyone else got any good ideas?#2018-11-2512:55borkdudeCommon mistake of mine:
$ clj -Srepro -Sdeps '{:deps {org.clojure/test.check {:mvn/release "RELEASE"}}}'
Error building classpath. nil
java.lang.NullPointerException
Maybe an error message could help here?#2018-11-2617:41rickmoynihanDoes anyone know if this issue likely to be resolved?
https://dev.clojure.org/jira/browse/TDEPS-82
I think it causes lein-tools-deps (which first uses -Sdescribe) to resolve dependencies unnecessarily and too soon#2018-11-2618:50alexmillerYes, would like to fix#2018-11-2620:01seancorfieldI patched the clojure script locally and was hoping to provide a patch to that ticket but my weekends have not been conducive to that, unfortunately.#2018-11-2620:01seancorfield(I'm not sure my quick'n'dirty local patch is an ideal solution to submit yet)#2018-11-2620:27alexmilleryeah, sorry I haven’t had a chance to look at it yet#2018-11-2620:27alexmillerI’m hoping to have some cycles post-conj to catch up on tools.deps stuff#2018-11-2715:45adamfreycan :jvm-opts be put at the top level of a project's deps.edn file?
like:
:jvm-opts ["-Xms512M"
"-Xmx1g"
"-Djava.io.tmpdir=./tmp"
"-Dcatalina.base=./tmp"]
#2018-11-2715:46adamfreyIt doesn't seem to be working but I can't find an example in the docs so maybe that's not a supported feature#2018-11-2715:49adamfreyah nevermind, it doesn't exist yet. I found a discussion about the same thing from a few days ago: https://clojurians-log.clojureverse.org/tools-deps/2018-11-12#2018-11-2716:00dominicmI guess it doesn't make much sense for that to exist does it?#2018-11-2716:03adamfreywhy doesn't it make sense? I would like to start a a clojure process from my project and have a bunch of jvm opts applied without having to specify them in the cli each time.#2018-11-2716:03adamfreyThere's a JIRA ticket for it already: https://dev.clojure.org/jira/browse/TDEPS-63#2018-11-2716:10dominicmbecause it works from an alias#2018-11-2716:15dominicmI'm thinking about the expectations I have, which is that :aliases is ignored from deps.edn transitively, and everything else is transitive.#2018-11-2720:39aisamuI'd say it follows the same logic as a root-level :main-opts#2018-11-2721:53dominicmThat doesn't exist either! #2018-11-2722:04aisamuPrecisely simple_smile#2018-11-2716:29eval2020Can one require deps in the init-script?#2018-11-2716:31eval2020This succeeds: clj -e '(require (quote [clj-time.core :as t])) (prn (t/now))' but I cannot make it work from init-script.#2018-11-2716:36eval2020I was doing weird stuff: yes this works as expected#2018-11-2718:39dominicmthought this was a need trick:
#!/bin/sh
deps="{}"
if [ "$#" -gt 0 ]; then
deps="$(clojure -A:user/find-deps "
I named it replme.#2018-11-2722:25winkywooster@dominicm that’s cool, thanks for sharing#2018-11-2810:56eval2020@dominicm tools-deps equivalent of lein-try - very nice! 👌:skin-tone-2:#2018-11-2812:33borkdudewhat does user/find-deps do?#2018-11-2812:34dominicm@borkdude delegates to https://github.com/hagmonk/find-deps#2018-11-2814:32eval2020Published something similar (ie rebel-readline + deps) earlier this year.
The above inspired me to actually document it: https://gitlab.com/eval/deps-try/tree/master#deps-try#2018-11-2816:28dominicmWhat's the :repl/try about?#2018-11-2820:28borkdudeIs there a way to get -R test + -C test + also the possibility to override -m? With -A:test I cannot do this.#2018-11-2820:30eval2020@dominicm it’s a hook from rebel that kickstarts the loading of the libs (calling (unreleased) tools-deps’ add-lib). (Must be an easier way to start a repl where you want some commands to be executed before taking input)#2018-11-2821:54dominicmThere is, edge does it#2018-11-2821:55dominicmRebel takes the same flags as clojure main#2018-11-2821:56dominicmhttps://github.com/juxt/edge/blob/master/main/aliases/rebel/edge/rebel/main.clj#2018-11-2822:26kennyIs there a way to get rid of the logback.xml file that aws-maven brings in via tools-deps?#2018-11-2822:29dominicm@kenny I solved it by making a fork and opening a jira.#2018-11-2822:31dominicmhttps://github.com/SevereOverfl0w/tools.deps.alpha/tree/add-lib?files=1 I based it on the add-lib branch. I can live without s3 wagon support. #2018-11-2822:31kennyUnfortunately I need S3 support because Datomic Cloud stores its artifacts there 😞#2018-11-2822:32dominicmhttps://dev.clojure.org/jira/browse/TDEPS-108#2018-11-2822:34alexmillerI am interested in fixing that for you, after conj (although there may be new ways to do so)#2018-11-2906:09dominicmThat's great, thanks!#2018-11-2822:34dpsuttonwould make a fun unsession to fix a bug with alex miller#2018-11-2822:37alexmillerOh yeah so fun ;)#2018-11-2822:37dpsutton"work during the conference. thanks" lol#2018-11-2822:51mattyulrichHi all. I'm having an issue that looks like transitive dependency problems with a corporate repo. I have the repo listed in :mvn/repos - but it looks like transitive deps from java libraries that I reference from that repo are not resolving. When trying to resolve the deps, it seems to be looking at clojars instead. Is this a known issue or something that should be working?#2018-11-2822:55hiredmanyou might checkout the poms of those transitive deps to see if they do anything kind of odd like relocations https://dev.clojure.org/jira/browse/TDEPS-8#2018-11-2822:57hiredmanthere are a lot of issues in the tools.deps jira, so in order to discern you are experiencing a known issue or not, you may have to narrow it down#2018-11-2823:20amarHi everyone. Has anyone been able to use a private git library as a dependency in a CI environment like Google Cloud Build? The key is there and if I use git from the command line, the repo can be successfully cloned.
I'm seeing
com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available:
at com.jcraft.jsch.agentproxy.ConnectorFactory.createConnector(ConnectorFactory.java:120)
at clojure.tools.gitlibs.impl$fn__843.invokeStatic(impl.clj:30)
at clojure.tools.gitlibs.impl$fn__843.invoke(impl.clj:28)
at clojure.lang.Delay.deref(Delay.java:42)
#2018-11-2906:10dominicmYou need to start an ssh agent, and add the key to it#2018-11-2921:06amarGreat. Thanks!#2018-11-2900:00mattyulrichThanks @hiredman - don't see a dependency relocation, but it does look like a good list of Jiras to review. I'll dig into there and see if I can see my issue.#2018-12-0117:59pesterhazyLooks like clj main-opts don't compose. For example, I tried to define two aliases that both set main-opts: https://github.com/pesterhazy/advent2018/blob/e0f8fba23339052eb112958fed712da97c7c270b/deps.edn#L3#2018-12-0118:01pesterhazyUnfortunately with clj -A:nrepl:in-ns, the -e (...) argument for the second alias overrides the -e (...) of the first#2018-12-0118:01pesterhazyIs this intended behavior? Is there another way for an alias to execute code?#2018-12-0118:02pesterhazyNote that clj -A:nrep -e (...) does work#2018-12-0118:21dominicmNon composition is expected. #2018-12-0118:26pesterhazyThose are two enhancements to clj that would make aliases more useful then: avoiding the need for Corfield Commas, and being able to stack side effects (like starting an nREPL server) on top of each other#2018-12-0121:46alexmillerYou can stack effects with -m - main args are appended to the end. But, there are other things going on in your example above (no support for -e aliases, ordering of -A vs other aliases, etc)#2018-12-0211:18dominicmhttps://clojure.org/reference/deps_and_cli
> If multiple -M alias maps are activated, only the last one will be used#2018-12-0211:23pesterhazyso in this case it's better to use -M:alias1:alias2 than -A:alias1:alias2?#2018-12-0211:48dominicmThe former would have no meaning #2018-12-0211:48dominicmWell, the former is equivalent to -M:alias2#2018-12-0211:49pesterhazyOk, I'll give that a try#2018-12-0211:50dominicmI opened an nrepl issue to start a discussion about how this might work. I've not come up with anything good. #2018-12-0211:58pesterhazy@U09LZR36F can you link to the issue?#2018-12-0212:05dominicmhttps://github.com/nrepl/nrepl/issues/54#2018-12-0212:05dominicmI should update it to mention the actual problem statement#2018-12-0212:05dominicmI did think about a main Hopper #2018-12-0212:06dominicmA simple version could be put together in an afternoon#2018-12-0214:27alexmillersorry, I meant you could stack effects from last alias + command line, not across aliases. I think making ordering do stuff beyond that is too fragile#2018-12-0214:28alexmillerthe order of “last” should be unambiguous (but is currently not, and that’s a bug (https://dev.clojure.org/jira/browse/TDEPS-100 which may be irritating to fix)#2018-12-0214:29alexmillerat some point, you are putting too much on CLJ and should recognize that you are writing a program and push this complexity into a Clojure program I think#2018-12-0214:29alexmillerbut I’m willing to consider that I’m wrong on that :)#2018-12-0215:06dominicmIt's a little bit complicated, because I think mains have been pushed as a common interface. But they don't compose exactly.
But anything you want to do, can be done with jvm opts and a custom main.#2018-12-0321:26pesterhazyclojure -Sdeps '{:aliases {:a1 {:main-opts ["-e" "(println,:a1)"]} :a2 {:main-opts ["-e" "(println,:a2)"]}}}' -Srepro -M:a1:a2 will print out only :a2#2018-12-0321:26dominicmCorrect.#2018-12-0321:27dominicmBest bet is a custom main, and use jvm opts to compose.#2018-12-0321:27pesterhazyAlso you can only ever specify one -m/--main option#2018-12-0321:28pesterhazyTo me, those two facts combined diminish the usefulness of aliases, because you often need to produce a side effect to make an alias effective#2018-12-0321:28pesterhazyAs an example, to start an nREPL server, you need to call nREPL's start-server function#2018-12-0321:29dominicmYep! This is a limitation from Clojure itself, inherited from Java. I've given this some thought, and I don't know what multiple mains look like.
In parallel presumably? #2018-12-0321:29pesterhazySo one person in the team uses nREPL so they, so they run,e e.g. clojure -M:nrepl#2018-12-0321:31pesterhazyThat to me is how composition of aliases would help - in different situations you can enable different sets of features#2018-12-0321:32pesterhazyUsing :main-opts with ["-e" "(do (require ...) (start-server))]" is a little crude but works-ish - except for the fact that main-opts don't compose#2018-12-0321:33dominicmYou can call functions, as long as they exit, but nrepl main doesn't, else it would be fairly useless as a main#2018-12-0321:33pesterhazyIf not through aliases, I'm not sure how I would accomplish the same thing#2018-12-0321:34dominicmEdge has a workaround for this already, it looks at the system properties, and loads things based on flags there.#2018-12-0321:35pesterhazydo jvm opts compose?#2018-12-0321:35dominicmYep! That's what I was trying to get across 😊 #2018-12-0321:36pesterhazyGotcha. It seems a little arbitrary though that jvm-opts get concatenated, whereas main-opts are replaced#2018-12-0321:39dominicmJvm opts naturally concatenate in the underlying system. I think they're unordered.
Main opts is talking about clojure's main-opts though.
You wouldn't be able to do what you want with nrepl and other mains even if concatenation happened. #2018-12-0321:39pesterhazywhy?!#2018-12-0321:40pesterhazythis does work https://github.com/pesterhazy/advent2018/blob/e0f8fba23339052eb112958fed712da97c7c270b/deps.edn#L6#2018-12-0321:41pesterhazyis the problem with concatenating main-opts that the ordering of different aliases is not well defined?#2018-12-0321:43pesterhazyhere's where the behavior is defined btw https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha.clj#L26#2018-12-0321:52pesterhazy@U09LZR36F I guess your idea is to use an alias like {:jvm-opts ["-Dclojure.run.foo=foo.core"]} and then have a main that scans through system properties and calls those in some order?#2018-12-0322:01dominicm@U06F82LES somrthing like that, yeah#2018-12-0322:02dominicm@U06F82LES nrepl.main blocks forever. So rebel would not run afterwards.#2018-12-0322:10pesterhazynrepl.server/start-server doesn't block#2018-12-0322:11dominicmYou're thinking of clojure mains opts, I'm thinking of plain mains. My mistake.#2018-12-0322:12dominicmI don't think you can pass -e and -m anyway.#2018-12-0400:08alexmillerSomething that we have discussed is a “start” option that would invoke an arbitrary function#2018-12-0118:28pesterhazyCorfieldization is a fun exercise but a clearer way to achieve this effect would be great#2018-12-0118:57dominicmThe comma thing is a bug afaik#2018-12-0120:50seancorfieldI'm sure that if someone can come up with patches to the clojure shell script that eliminate Corfield, lots of people will be very happy. Me included.#2018-12-0121:12dominicmFixing it is difficult I would guess. You have to figure out how to generate properly escaped strings from edn.#2018-12-0121:27seancorfieldMainly you need to be able to round-trip arbitrary strings from bash to file to bash again.#2018-12-0121:27seancorfieldPortably 🙂#2018-12-0121:41dpsuttonWill the corfield comma work in a powershell implementation of clj?#2018-12-0121:44alexmillerSean’s description is pretty good, happy to have someone figure that out#2018-12-0121:44alexmillerI suspect powershell is prob easier#2018-12-0121:49dottedmagAt least it's bash, not an arbitrary POSIX shell.#2018-12-0121:54alexmillerOne issue is with nested quotes, another is needing to write the second half in bash (first half is in Clojure)#2018-12-0121:55dottedmag@alexmiller if you can give me an example of what needs to be done, I'll try to stretch my shell muscles.#2018-12-0122:25alexmillerSorry, boarding a flight but there is a ticket in TDEPS#2018-12-0122:27seancorfieldhttps://dev.clojure.org/jira/browse/TDEPS-56#2018-12-0122:27seancorfield(there may be others but that's the first one I found)#2018-12-0122:29seancorfield@dottedmag ^ yup, that seems to be the whole issue.#2018-12-0123:23pesterhazyI’ll take a look as well#2018-12-0211:04pesterhazyI'm trying to tackle TDEPS-56, but am having trouble building the project. I've discovered two repos, brew-install and tools.deps.alpha. In the former I can run cd src/main/resources && bash -x install.sh ~/tmp
but that fails because *.jar is missing#2018-12-0211:06pesterhazyPresumably the jar is built from the tools.deps.alpha repo, but am I supposed to copy that over to brew-install/src/main/resources ?#2018-12-0211:21pesterhazyNext up, I'm running into this issue: bin/clojure: line 174: $install_dir/libexec/clojure-tools-${project.version}.jar: bad substitution#2018-12-0211:22pesterhazy${project.version} is not valid Bash syntax so that must be related to the maven "filtering" feature#2018-12-0211:43pesterhazylooks like I was confused - master in the brew-install repo is an outdated branch#2018-12-0214:08pesterhazyok figured it out - in a nutshell you need to go through the package.sh script and before that mvn install tools.deps.alpha (also point the pom to the snapshot version you're testing)#2018-12-0214:31alexmilleryes, exactly#2018-12-0214:09pesterhazyAdded a patch: https://dev.clojure.org/jira/browse/TDEPS-56?focusedCommentId=50847#comment-50847 cc: @seancorfield @dottedmag#2018-12-0317:38seancorfield@U06F82LES I share Alex's concern about both the eval aspect and the lack of compatibility with existing cached .main files -- I wonder if the script could verify the file contents, ensuring it was all \c\h\a\r\a\c\t\e\r\s and only calling eval in that case? I think that would retain compatibility with existing files and also address the security issue? /cc @U064X3EF3#2018-12-0317:43alexmillerI think we should not use eval. I think probably it would be best to read the file as an array and then array splat it into the command line. I think that largely fixes the spacing problem, but not sure what still needs to be solved for quoting.#2018-12-0317:44pesterhazyhappy to try a different tack#2018-12-0317:45pesterhazyto read the file into an array from dash, we'll need to use a separator#2018-12-0317:45pesterhazywere you thinking of separating arguments by newline?#2018-12-0317:46alexmillerdo you need a separator? I thought space was the separator#2018-12-0317:47pesterhazyright, then we'll need to escape space#2018-12-0317:48pesterhazyso we'd write this -co {:aot-cache\ true}
#2018-12-0317:49alexmillerdo you? the whole idea here is to write the string with spaces, then read it back into an array (breaking on spaces), then drop that into the args list#2018-12-0317:50alexmillerI think that’s actually fine and the issue is really what happens when you have Clojure quoted things inside the args#2018-12-0317:50pesterhazyhere's how I'm thinking it could it could work:
- in the clojure code, we'd replace " " with "\\ " and "\\" with "\\\\"
- in the bash code, we'd read the file char by char, replace "\\ " with a space, replace "\\\\" with "\\" and else when encounter an unescaped space, start a new argument#2018-12-0317:51pesterhazywe'd basically implement our own quote/unquote scheme#2018-12-0317:52pesterhazythe main point is that space-as-arg-separator and space-as-literal-space need to be distinguished in the bash code#2018-12-0317:56alexmillernewline separator might also be a better choice - I think you may be able to automatically get some of what we want by changing the IFS too#2018-12-0317:57alexmillerideally, I think the main file should have as little markup as possible#2018-12-0317:57pesterhazysure, though the main-opts strings may contain newlines as well#2018-12-0317:58alexmillerthe less magic that’s in the Clojure code the better. we’re just creating portability problems the more coordination is required between these two pieces of code#2018-12-0317:58pesterhazyagreed#2018-12-0317:58pesterhazymaybe keep a Windows version in mind as well#2018-12-0317:58alexmillerI think new lines in the args is much less common than multi-word / quoting#2018-12-0317:59alexmillerI am keeping Windows in mind :) (I suspect powershell has better options for this stuff though)#2018-12-0318:00alexmillerI need to do some other stuff, checking out#2018-12-0318:00pesterhazyeven newlines aren't common, it shouldn't break if they do appear#2018-12-0318:00pesterhazyi'll give it another try#2018-12-0319:17pesterhazyUsing newlines as arg separators, I wrote an explode Bash function that takes a string and generates an array: https://github.com/clojure/brew-install/pull/3/files#diff-b3212e45e19f61de4754a755466b793f - it allows you to escape newlines by prefixing them with a backslash. So the file format is simple.#2018-12-0319:17pesterhazyIf that looks like the right direction, I'll prepare a new patch#2018-12-0314:37jeroenvandijkI'm having an issue creating an uberjar using tools.deps with boot-clj via boot-tools-deps. The clj files of my local and git dependencies are not included if I don't AOT these files. Is this the right place for such an issue?#2018-12-0314:38jeroenvandijkI've created an example project with the issue here https://github.com/jeroenvandijk/tmp-boot-tools-deps-issue#2018-12-0314:38jeroenvandijkThe files https://github.com/jeroenvandijk/tmp-boot-tools-deps-issue/blob/master/uberjar.txt#L21-L23 are in the fileset, but not in the jar#2018-12-0316:19seancorfieldMy advice would be to use one of the uberjar utilities designed for deps.edn instead of Boot. #2018-12-0316:22seancorfieldboot-tools-deps was intended as a simple bridge for starting with a Boot project and moving the primary dependencies into deps.edn - there are several inherent limitations of the approach. #2018-12-0316:23seancorfieldAt work, we moved off Boot completely. #2018-12-0316:23seancorfield@jeroenvandijk ^#2018-12-0316:30jeroenvandijk@seancorfield Ah good to know, thanks 😬. Out of curiosity what are you using for uberjars?#2018-12-0316:31dominicm@jeroenvandijk I'll plug pack.alpha which I authored. Sean maintains a depstar fork.#2018-12-0316:33jeroenvandijk@dominicm cool thanks. What's the best choice for creating uberjar's? Capsule, onejar?#2018-12-0316:34jeroenvandijk(btw I tried pack.alpha, but it seems to force me to choose a main namespace as jar entry. Something i'm not used to with lein and boot-clj)#2018-12-0316:35dominicm@jeroenvandijk capsule works in more contexts. I'm surprised you say that, because pack works on the opposite, clojure.main is the default unless you set it.#2018-12-0316:36jeroenvandijkok i'll have to try again 🙂 Maybe i gave up too quickly#2018-12-0316:36jeroenvandijkThanks, i'll come back with the results tomorrow#2018-12-0316:37dominicm@jeroenvandijk feel free to bother me if something doesn't work.#2018-12-0316:38jeroenvandijkawesome, thanks!#2018-12-0316:52seancorfield@jeroenvandijk My fork of depstar -- https://clojars.org/seancorfield/depstar#2018-12-0316:58jeroenvandijkthanks!#2018-12-0316:53seancorfieldWe create all our production artifacts with that (and run them with java -cp path/to/the.jar clojure.main -m entry.point for them to start at entry.point/-main).#2018-12-0412:30borkdudedoes tools.deps support git deps that have an edn file in a subdirectory? do you have to reflect this in the git url?#2018-12-0412:30borkdudee.g. my-project/lib2, my-project/lib2#2018-12-0412:30dominicm@borkdude you can use :deps/root to specify a sub-directory#2018-12-0412:30borkdudeperfect!#2018-12-0417:32shaun-mahoodSpecifically for use with REBL and openjdk, is there a way to tell REBL bring in the correct version of JavaFX as part of the extra-deps? I don't actually need it at the moment (I switched to the Java version to get it to work), but it seems like it would be useful to prevent problems (I ran into it on 2 of my machines that I use regularly for Clojure work).#2018-12-0417:33seancorfieldMaybe ask in the #rebl channel?#2018-12-0420:48dottedmagIt's a pity that existing .main-files are lossy: both arguments are separated by spaces and spaces within arguments are written as spaces.#2018-12-0420:49dominicm@dottedmag you mean via :main-opts?#2018-12-0420:49dottedmagAny idea how to invalidate those? I could use readarray Bash builtin to read \0-delimited file into a Bash array.#2018-12-0420:49dottedmag@dominicm yes.#2018-12-0420:52dottedmagHmm, I can change $ck, but that invalidate all existing cached files, no matter if :main-opts had had spaces in them or not.#2018-12-0420:53dottedmag@alexmiller ^ would that be acceptable?#2018-12-0420:55dominicmhttps://dev.clojure.org/jira/browse/TDEPS-56 🙂 some work is happening now. Ideas are being thrown around. \0 delimiters might be interesting, but can't parameters have \0 in them?#2018-12-0420:56dottedmag@dominicm That's the ticket I'm working on. No, Bash cannot keep \0 in variables or arguments.#2018-12-0420:56dottedmagThere is even a hack in Bash to use '' as a "delimiter" to denote "use \0 as a delimiter".#2018-12-0420:56dominicmDidn't realize you were down in the trenches with this 🙂#2018-12-0420:57dottedmagAlso, execve(2) API accepts NUL-terminated strings, so no way \0 can be passed through.#2018-12-0420:58dottedmagWell, the patch itself is not so complicated. The only remaining issue is backward compatibility with previously written .main files.#2018-12-0420:59dominicmOn a personal note, it seems to me that updating clj may/should invalidate all my caches#2018-12-0421:07alexmillerit is supposed to, as the install-level deps.edn is included in the cache calculation#2018-12-0421:08alexmillerbut “invalidate” != “delete”#2018-12-0421:08alexmilleras the idea of writing a script that goes around rm’ing your dirs scares the crap out of me#2018-12-0421:09alexmillerusing a .main2 seems perfectly acceptable too#2018-12-0421:15timgilbertI wonder if this would be usable for the string-splitting problem: http://mywiki.wooledge.org/BashPitfalls#hosts.3D.28_.24.28aws_....29_.29#2018-12-0421:16timgilbert...or if it would be acceptable to fall back to a tool written to handle this kind of thing, like awk or something#2018-12-0421:18timgilbert...or if the string could be written to .main separated by commas, and then read into the array on the bash side via readarray -d ,#2018-12-0421:43alexmillerwould prefer to a) not add any additional deps if at all possible and b) have the clojure program write something as dumb and obvious as possible to the .main file#2018-12-0421:56dottedmag@alexmiller Only locations of install-level deps.edn files are included in the key of cache, not their contents, right?#2018-12-0421:57pesterhazyi'd like to point out that I've put some thought into this and proposed a solution for quoting: https://github.com/clojure/brew-install/pull/3/files#diff-b3212e45e19f61de4754a755466b793f#2018-12-0421:57pesterhazy(and submitted a first pass at a patch)#2018-12-0421:58dottedmag@alexmiller How about this one? Or I can do .main2, it's even simpler.#2018-12-0421:59pesterhazyif I'm not on the right track, happy if others take a swing at this, but I wouldn't want effort needlessly wasted if possible#2018-12-0421:59pesterhazydiscussion here: https://clojurians.slack.com/archives/C6QH853H8/p1543759744032700#2018-12-0422:00dottedmag@pesterhazy Seems much easier to do \0-based output and read back using readarray like this: - main_cache_opts=($(cat "$main_file"))
+ declare -a main_cache_opts
+ readarray -d '' -t main_cache_opts < "$main_file"
#2018-12-0422:00dottedmagThe only remaining change is to ignore .main and and read/write .main2 instead.#2018-12-0422:01dottedmag@pesterhazy t.d.a change is also trivial: --- a/src/main/clojure/clojure/tools/deps/alpha/script/make_classpath.clj
+++ b/src/main/clojure/clojure/tools/deps/alpha/script/make_classpath.clj
@@ -76,7 +76,7 @@
(when (.exists jf)
(.delete jf))))
(if-let [main-opts (seq (get (deps/combine-aliases deps-map (concat aliases main-aliases)) :main-opts))]
- (io/write-file main-file (str/join " " main-opts))
+ (io/write-file main-file (str/join "\u0000" main-opts))
(let [mf (jio/file main-file)]
(when (.exists mf)
(.delete mf))))))#2018-12-0422:02dottedmag@pesterhazy I have to go right now, so I can't finish it at the moment.#2018-12-0422:03alexmillersorry, I don’t have time to work on this atm but will hopefully have a slice of deps time in a day or two#2018-12-0422:03dottedmag@pesterhazy I don't mind if you pick it up from here, if you think this approach is good enough.#2018-12-0422:04alexmillerif there are two (or N) options, that’s fine, just describe the tradeoffs in a table (and separate the file impl aspect from the “not breaking existing stuff” aspect - those I think are orthogonal)#2018-12-0422:07pesterhazythe obvious downside of readarray is that it's not available in bash3, which is what ships with macOS#2018-12-0422:07pesterhazyI think that rules it out#2018-12-0422:08dottedmagOh, too bad.#2018-12-0422:10pesterhazyalso not sure how reliable reading NUL character in bash is - that's the kind of thing where bash if flaky#2018-12-0422:11pesterhazyaccording to this https://stackoverflow.com/a/6571549/239678: "In Bash, you can't store the NULL-character in a variable."#2018-12-0422:12dottedmagThat's why they have readarray in first place.#2018-12-0422:12pesterhazymany languages with C strings have trouble with NUL characters in strings, for the obvious reasons#2018-12-0422:12dottedmagAnd special case for -d '' there#2018-12-0422:13dottedmagread -r -d '' should work with an explicit loop.#2018-12-0422:13dottedmagAs of now all NUL-characters are stripped from .main when read by Bash, so no backward compatibility problems here.#2018-12-0422:14dottedmagUnless someone relied on NULs being stripped 😄#2018-12-0422:14pesterhazywell try it - I didn't have luck reading the "\\ " character sequence using read#2018-12-0422:14pesterhazyno it's not backward compatible - it would turn potentially multiple arguments into a single argument, i.e. change the semantics#2018-12-0422:15pesterhazyI don't think there's a way around using a different file extension#2018-12-0422:16dottedmag@pesterhazy $ echo '00000000: 310032003300' | xxd -r | while read -r -d '' i; do printf ">$i<\n"; done
>1<
>2<
>3<
#2018-12-0422:18dottedmag@pesterhazy By backward compatibility I meant backward compatibility for code that relied on \u0000 passed from deps.edn unchanged to the app. But that didn't work, so we don't need to care about it.#2018-12-0422:27pesterhazy@alexmiller I've updated the ticket with a summary#2018-12-0423:41dominicmThe ticket doesn't mention back porting readarray, https://stackoverflow.com/a/49322365 has some hints. Would this work? #2018-12-0503:50baritonehandsI created a PR for more Deps/CLI docs: https://github.com/clojure/clojure-site/pull/315#2018-12-0503:50baritonehandsnot sure what the process is for evaluating these#2018-12-0503:52seancorfieldMy simple guide for AOT: do not do it.#2018-12-0503:52seancorfieldAOT is both unnecessary and a giant pain in the ...#2018-12-0503:57seancorfield(for all my snarkiness, the tip about needing to create classes and having it on your classpath is a good one -- it bit me when I tried to manually compile stuff)#2018-12-0503:58baritonehandsas much as it is not preferred, it is still necessary in some cases#2018-12-0503:58seancorfield@gfredericks mentioned the classes thing in his talk at Conj -- I think it's a widely not-known thing 😞#2018-12-0503:59seancorfieldYeah, AOT is a necessary evil in some very rare cases. Knowing how to successfully compile stuff when building JAR files, for example.#2018-12-0504:00seancorfieldDon't worry, I'm just very grumpy about AOT 🙂#2018-12-0504:02baritonehandsAh @gfredericks fellow Chicagoan#2018-12-0504:17seancorfieldHave you watched his talk from Conj? Definitely worth it.#2018-12-0504:18baritonehandsNot yet, still working my way through all of them#2018-12-0504:18baritonehandsgot distracted by #adventofcode lately#2018-12-0504:19seancorfieldAh, I've avoided that every year. https://www.youtube.com/watch?v=-Qm09YiUHTs&index=25&list=PLZdCLR02grLpMkEBXT22FTaJYxB92i3V3&t=0s for the direct link. Really good stuff!#2018-12-0609:28kirill.salykinhi,
I am trying to build skinny jar with pack and got this error:
Exception in thread "main" java.lang.IllegalArgumentException: No matching clause: [nil :jar]
at mach.pack.alpha.skinny$run_steps.invokeStatic(skinny.clj:107)
at mach.pack.alpha.skinny$run_steps.invoke(skinny.clj:99)
at mach.pack.alpha.skinny$_main.invokeStatic(skinny.clj:175)
at mach.pack.alpha.skinny$_main.doInvoke(skinny.clj:158)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
#2018-12-0609:28kirill.salykinbut jar is created#2018-12-0609:29kirill.salykinplease advice what I am doing wrong#2018-12-0609:29kirill.salykinpack version:
{pack/pack.alpha {:git/url ""
:sha "ddbbdbd4001c75647deddaad0f8bb7dc954fcb6f"}}
#2018-12-0609:32kirill.salykinUPD: I switched to latest version in master and everything seems to work#2018-12-0609:36kirill.salykinone thing tho - skinnyjar stucks if target directory contains prev version…#2018-12-0609:41dominicm@kirill.salykin master is a pretty big rewrite of pack. This bug isn't terribly surprising 😊
What do you mean by "contains previous version"?#2018-12-0609:41kirill.salykinif you run it twice in a row#2018-12-0609:41kirill.salykinrm -rf ./target/*
clojure -A:pack mach.pack.alpha.skinny
#2018-12-0609:42kirill.salykinthis fixes it#2018-12-0609:42dominicm@kirill.salykin what is the output in the case that it gets stuck? I expect a thread will error something? #2018-12-0609:42kirill.salykinno output, just takes a lot of time#2018-12-0609:42kirill.salykinmore then 1 minute on machine#2018-12-0609:42kirill.salykini was not patient to wait longer…#2018-12-0609:42dominicmHmm. I'll try and debug that today. I'm not sure what could cause that.#2018-12-0609:43kirill.salykinif you clean the target directory - everything works perfectly#2018-12-0609:43dominicmMaybe there's something accidentally recursive.#2018-12-0609:45kirill.salykinshall I create the issue?#2018-12-0610:22dominicmFeel free, I'll try and get round to it today 🙂#2018-12-0610:23kirill.salykinwill do, thanks for your help!#2018-12-0610:42dominicm@kirill.salykin I'm unable to reproduce what you're seeing 😟 I might need some more context, maybe it only happens under certain deps.edn files or something?#2018-12-0610:42kirill.salykinI’ll try to create some minimal repro case#2018-12-0610:43kirill.salykinthanks for looking#2018-12-0610:43dominicmI'll mirror this on github 🙂 Not sure which channel you prefer for tracking this#2018-12-0610:44dominicmfwiw, I tested against pack itself like this: clojure -m mach.pack.alpha.skinny --lib-dir /tmp/lib --project-path skinny.jar#2018-12-0610:47kirill.salykinmaybe it is something in our project#2018-12-0610:47dominicmWhat does your :pack alias look like?#2018-12-0611:01kirill.salykinhm
:pack {:extra-deps {pack/pack.alpha {:git/url ""
:sha "4a00565dce29225a1dd2a810c438b90b14f9e111"}}
:main-opts ["-m"]}}}
#2018-12-0611:01kirill.salykinI am bit confused with empty main-opts#2018-12-0611:02kirill.salykinlemme try w/o main-opts at all#2018-12-0611:02dominicm@kirill.salykin it really just saves you from typing -m 😄#2018-12-0611:02kirill.salykinso it doesnt matter?#2018-12-0611:03dominicmnot particularly 🙂#2018-12-0611:11kirill.salykinworks on empty project
I assume it is something within my project#2018-12-0611:36dominicmif you can track down what's within your project, I'd be interested#2018-12-0612:01kirill.salykinI’ll try, but later. It seems not urgent, also it is nice to clean target directory anyway (for outdated libs for instance)
so not with rm -rf target/* it just works#2018-12-0612:02kirill.salykinbtw, pack dies if target directory not created#2018-12-0614:16kirill.salykinQuestion regarding juxt/pack, do I understand correctly that using --lib-type keep with skinny-jar should keep clojure file in jar?
the issue I see now that git dependency packed as empty jar (there is no gen-class) and clojure complaints that can find file on classpath#2018-12-0614:17dominicm@kirill.salykin --lib-type keep means that any dependencies that are jars, will stay as jars, and git dependencies which are directories, will stay as directories.#2018-12-0614:19kirill.salykinis it by design that git dependency packed into empty jar? or do I missing something in config?#2018-12-0614:20dominicmthat sounds wrong. I haven't seen that in my testing#2018-12-0614:21kirill.salykinwill try to create something that can reproduce the issue#2018-12-0614:21kirill.salykinthanks!#2018-12-0614:25dominicm@kirill.salykin my mistake, I've reproduced#2018-12-0614:25dominicmworking on it#2018-12-0614:25kirill.salykinThanks!#2018-12-0614:27mccraigmccraigi've got a lein-modules based project in a monorepo which has common deps in the top-level project.clj and per-module deps in each module subdirs own project.clj. it builds and tests all modules in a sensible (poset) order and correctly outputs versioned jars for each module - is there a way of achieving similar behaviour for deps.edn based project ?#2018-12-0614:27dominicm@mccraigmccraig what's poset order? I would just loop over each directory with bash & use pack 😄#2018-12-0614:28mccraigmccraigit figures out the correct order itself, based on the dependency graph#2018-12-0614:28dominicm@kirill.salykin I've figured it out, dumb typo#2018-12-0614:33dominicm@kirill.salykin Try 24863d1ff5a54bb7cc783ff62272212b51c8b316#2018-12-0614:35kirill.salykinwow, it blazingly fast, thanks!
trying#2018-12-0614:36dominicm@kirill.salykin I go on holiday soon, and I've wrapped up work 🙂 I have a few spare cycles.#2018-12-0614:38kirill.salykinyay! works! thanks#2018-12-0614:39dominicmGreat 🙂#2018-12-0615:09kirill.salykinbtw, stuck issue seems fixed as well#2018-12-0615:14dominicmoh, hmm 🤔#2018-12-0615:16dominicm¯\(ツ)/¯#2018-12-0619:13rpkarlssonI ran into some trouble using java9+ and Cider with deps.
Deps builds abbreviated paths ("src", as specified in :paths) instead of full paths ("/home/rpkarlsson/Projects/mc-kartan/src/") as other classpath builders seem to do.
Cider with java1.8 and lower determines the classpath in a different manner and is not effected by this.
Are abbreviated paths something that could potentially lead to other problems and because of that should be fixed in deps or should abbreviated paths be expected in tooling?
More info at:
https://github.com/clojure-emacs/cider/issues/2533#2018-12-0622:34dottedmagComments, sugestions? https://dev.clojure.org/jira/browse/TDEPS-56?focusedCommentId=50940&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-50940#2018-12-0622:35dottedmagI've got the spaces work in :main-opts#2018-12-0623:15timgilbertLooks reasonable to me, personally, though I'd prefer newlines to NUL characters.#2018-12-0623:40pesterhazy ¯\(ツ)/¯#2018-12-0623:41pesterhazyI’m not playing the game of uploading competing patches#2018-12-0623:45alexmillerif there are competing approaches, please summarize the tradeoffs#2018-12-0623:45alexmillerI need that to make decisions and I will have a slice of time to do so soon#2018-12-0701:05dottedmag@timgilbert Newlines -> escaping comes back. Not good.#2018-12-0717:17donaldballWhat tools do folk like to use to build and publish jar artifacts to maven repos, particularly private s3 maven repos?#2018-12-0717:31dominicmhttps://juxt.pro/blog/posts/pack-maven.html#2018-12-0717:50seancorfieldAt work we use https://github.com/seancorfield/depstar to build our deployment uberjars and I also use it now to build "thin" JARs to deploy on Clojars.#2018-12-0717:51seancorfieldFor the actual "push", I use mvn deploy:deploy-file ...#2018-12-0717:52seancorfield(the same, nasty invocation that @U09LZR36F has in his blog post, wrapped up in a shell script -- that also updates pom.xml with the SHA of the latest commit so that Clojars can link back to the commit that created the release)#2018-12-0717:57donaldballThanks for the pointers!#2018-12-0720:23timgilbertFWIW, I'm working on a tool to post-process the results of clojure -Spom to add in the groupId/artifactId and pull the version from a git tag, since checking in partly-generated stuff into source control bugs me#2018-12-0720:25timgilbert(with the goal of automating everything up to the jar creation and then using pack for the rest)#2018-12-0719:14harriganIf I create a pom.xml with the following dependency:
<dependency>
<groupId>org.lmdbjava</groupId>
<artifactId>lmdbjava</artifactId>
<version>0.6.1</version>
</dependency>
and run mvn dependency:tree I see lmdbjava with 10 transitive dependencies. If I create a deps.edn with the same dependency:
{:deps {org.lmdbjava/lmdbjava {:mvn/version "0.6.1"}}}
and run clj -Stree I see 9 transitive dependencies. com.github.jnr:jffi:jar:native:1.2.16:runtime is missing. Could this be an issue with tools.deps?#2018-12-0719:23seancorfieldSounds like https://dev.clojure.org/jira/browse/TDEPS-50 maybe?#2018-12-0719:28harriganAh that’s it exactly — thanks for the link.#2018-12-1014:10Brendan van der EsHi all, question: Suppose I'm in a REPL in the middle of a datomic session with large cache in memory and I need to pull in a new library to continue processing. Is 'add-lib' the only way to pull in this dependency without losing the in-memory state of the jvm session? In other words, is there any way to have the CLI preserve this session state in memory in between reboots for new deps? Assuming the answer to this is a negative and suppose 'add-lib' fails, any other ways to preserve datomic memory cache between repl reboots?#2018-12-1015:55Ben Hammondis https://docs.datomic.com/on-prem/valcache.html any good for you?#2018-12-1015:56Ben HammondI know its probably too late for this repl session ...#2018-12-1014:19alexmillerprobably a better question for #datomic but afaik, the answer is no#2018-12-1014:36Brendan van der EsYea, was afraid of that. I'm exploring ideas for a more datomic-centric repl experience but would need to make this kind of reloading seamless. Thanks for the quick response; will try #datomic.#2018-12-1015:03jeremysHello, did someone try to install clojure with nix on os x? I am not using homebrew atm and I was wondering if the clojure package from nix included the necessary stuff to run tools deps and cli#2018-12-1015:32alexmillerdon’t know - it’s not built or maintained by the core team#2018-12-1015:42dominicm@jeremys I don't know package it is, but one of them does include tools deps.#2018-12-1015:43jeremys@alexmiller Ok thx!#2018-12-1015:44jeremys@dominicm looking at it there https://github.com/NixOS/nixpkgs/blob/b352d47e42b5babe82d937b2d5c77476b663dd88/pkgs/development/interpreters/clojure/default.nix it fetches https://download.clojure.org/install/clojure-tools-${version}.tar.gz#2018-12-1015:47jeremysI am thinking it fetches the necessary tools, I was about to take a look at the brew formulae to get a sense where things might differ but the tool part of the url on clojure’s website and the deps mention in the nix package leads me to think it’s there. Thx.#2018-12-1015:47alexmillerseems fine. that’s a couple versions back but what it does is intentionally pretty dumb.#2018-12-1015:48alexmillerthe brew recipe is https://github.com/clojure/brew-install/blob/1.9.0/src/main/resources/clojure.rb but it mostly calls the install script https://github.com/clojure/brew-install/blob/1.9.0/src/main/resources/install.sh#2018-12-1015:49jeremysI’ll take a look! thank you.#2018-12-1016:26jeremysJust for information the nix package comes with the tools deps machinery.#2018-12-1022:41rschmuklerHey all 🙂 . Is it possible to use my deps.edn to point to a github repo that is using a leiningen project.clj? I naively tried pointing at it but got hit with an error regarding detecting the manifest. Makes sense, and I realize it might be out of the scope of what tools.deps is trying to achieve... but also feels quite useful#2018-12-1022:45seancorfieldtools.deps, which is what does the dependency resolution under the hood, doesn't know how to use a project.clj file. Perhaps open an issue against that repo and see if they'll add a deps.edn file?#2018-12-1022:46alexmilleror if it has no transitive deps, you can force it to be a deps.edn project by putting :deps/manifest :deps in the coordinate#2018-12-1022:47seancorfieldPresumably, you could do that anyway and just add that project's deps to your own deps.edn file anyway @alexmiller?#2018-12-1022:47alexmilleryes, that should work#2018-12-1022:47alexmillersneaky! :)#2018-12-1022:47rschmuklerThanks for the quick replies! Sadly, it's of almost no value but for its transitive dependencies (which I needed to update). I ended up just excluding its (outdated) dependencies and then manually updating the dependencies in my deps.edn#2018-12-1022:48rschmuklerThanks again for the fast responses#2018-12-1110:23borkdudeIs there an overview of how people can get a REPL in their editor for a deps.edn based project (without adding nREPL-specific deps to the deps.edn, so by setting up an alias in their profile)?#2018-12-1110:55orestisAh, you’re looking for a proper guide, not just an alias?#2018-12-1110:55orestisHere’s my cider-nrepl alias, FWIW: :cider-nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.4.5"}
cider/cider-nrepl {:mvn/version "0.18.0"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}#2018-12-1110:56orestisI start my REPL by doing clojure -A:cider-nrepl and then connect CIDER to it via cider-connect. I think that any nREPL-aware editor could do the same?#2018-12-1111:13borkdudeyeah, maybe I could just add that to the README#2018-12-1117:10richiardiandreaI do the same the ClojureScript one is very similar#2018-12-1117:11richiardiandreaMore info here: https://nrepl.xyz/nrepl/usage/server.html#2018-12-1117:12borkdudecool, that’s the page I was looking for#2018-12-1117:12borkdudeI guess it has to be CIDER agnostic since some people use VSCode or whatnot#2018-12-1117:14borkdudehttps://github.com/borkdude/advent-of-cljc/blob/master/README.md#dev#2018-12-1117:14richiardiandreaYep well cider is just a middleware. Also keep in mind that Calva for VSCode does use the cider Middleware#2018-12-1117:15borkdudeHonestly I don’t know how that all works, I just use it#2018-12-1117:15richiardiandrea:+1: #2018-12-1110:23borkdudeI’d like to link to something like that in https://github.com/borkdude/advent-of-cljc#2018-12-1120:04pesterhazy@borkdude The problem with using an alias that sets "-m" "nrepl.cmdline" is that it doesn't allow you to also specify -r or -m rebel-readline.main#2018-12-1120:05pesterhazyTo me an nREPL-connected editor is only one part - the other part is a terminal/rebel-readline based prompt#2018-12-1120:06pesterhazyThat could be solved if nrepl.cmdline supported a --next-main or --chain argument, like --chain rebl-readlin.main. What it would do is to spawn the server in the background and then call the next main function#2018-12-1120:07pesterhazyIt could even do clean-up when that chained main exits#2018-12-1120:08pesterhazy@alexmiller has mentioned a start option, which could be in the same area#2018-12-1120:08orestis+1 for that. I never got to try it but I think you can combine -e to evaluate something arbitrary with -m ?#2018-12-1120:08pesterhazyyes you can -e something -e something-else -m some.main, though -m must be last#2018-12-1120:09pesterhazybut those don't work if the settings are specified in different aliases - the latter alias will override the earlier one#2018-12-1120:33viestiA colleague of mine ran into this: https://github.com/gfredericks/user.clj, maybe one could start nrepl server at load of user.clj? #2018-12-1120:41pesterhazyyes, a library could be used for this purpose#2018-12-1121:06borkdude@pesterhazy cool, if you have anything better to link to, I would update the README with it#2018-12-1121:17dominicmIn edge I have this working #2018-12-1121:17dominicmhttps://github.com/juxt/edge/blob/master/bin/rebel#2018-12-1121:18dominicmIt depends on a process in user.clj which searches for jvm options like load_xyz, which are then translated to namespaces to load. This means you can start arbitrary background services. #2018-12-1201:57potetmI write a clj script and wrap it w/ a bash script: https://github.com/potetm/advent-of-code/tree/master/bin#2018-12-1201:59potetmYou can put those on your path, add namespaced aliases to your ~/.clojure/deps.edn, and you can run nrepl from any dir containing a deps.edn.#2018-12-1216:12timgilbertSay, what directories do I need to cache in between invocations to take advantage of tools.dep's caching? I'm running builds on circleci and I'm caching ~/.m2, ~/.gitlibs, and .cpcache after running clojure -A:foo -Spath, but even after the cached directories are restored, when it invokes clojure -A:foo I see it downloading a bunch of jars from clojars.#2018-12-1216:15timgilbertOh, also caching ~/.clojure which has its own .cpcache#2018-12-1216:33dominicmM2 & gitlibs should be enough#2018-12-1216:36alexmillerthe ~/.clojure/.cpcache is only used if you are running clj without a deps.edn, so probably doesn’t apply#2018-12-1216:36alexmillerand you would need the local .cpcache if you want to avoid the second pre-jvm hit of building the classpath#2018-12-1216:37alexmillerbut all of those file checks are based on timestamps, so when you say “restored” I wonder if that changes the timestamps#2018-12-1216:37alexmillerparticularly in relation to the deps.edn file#2018-12-1216:54timgilbertHmm, yeah, not certain how it works. Maybe I can "negative touch" the deps.edn file to decrease its last-modified time#2018-12-1217:15alexmillereasier to touch the cpcache before you try to use clj?#2018-12-1217:15alexmillerI guess we could have a flag that was the opposite of -Sforce#2018-12-1217:16alexmillerthat would use the cached cp regardless of the timestamps#2018-12-1217:17alexmilleralthough then you’d have to make sure you did something else to force any deps changes to be picked up#2018-12-1217:42timgilbertJust confirmed on circleci that my deps.edn winds up being newer than the .cpcache and .m2 directories. I presume that circleci saves the mtimes on cache and restores them on restore. I'll try recursively touching those directories as an initial workaround#2018-12-1218:50hiredmanI dunno, the touch thing will likely work, but I am suspicious of deps.edn always being newer and you always download jars#2018-12-1218:51hiredmaneven if those files are older, it shouldn't cause you to re-download jars into your m2#2018-12-1218:57hiredmanyeah, if you are re-downloading jars over and over in to m2 that seems like the up stream issue and fixing it will take care of down stream#2018-12-1218:59hiredmanI dunno what facilities circleci provides, but you might stick a ls -R ~/.m2 at the start of your build before running clojure#2018-12-1219:28timgilbertHmm, the touch doesn't seem to have any effect:
ls -lRt --time-style=full-iso deps.edn .cpcache ~/.m2 || true
-rwxr-xr-x 1 circleci circleci 1660 2018-12-12 19:25:36.984493358 +0000 deps.edn
.cpcache:
total 36
-rw-r--r-- 1 circleci circleci 7989 2018-12-12 19:25:38.420533085 +0000 2750907078.cp
-rw-r--r-- 1 circleci circleci 24551 2018-12-12 19:25:38.420533085 +0000 2750907078.libs
-rw-r--r-- 1 circleci circleci 64 2018-12-12 19:25:38.420533085 +0000 2750907078.main
/home/circleci/.m2:
total 4
drwxr-xr-x 15 circleci circleci 4096 2018-12-12 19:25:38.380531978 +0000 repository
/home/circleci/.m2/repository:
total 52
drwxr-xr-x 3 circleci circleci 4096 2018-12-12 19:25:38.388532199 +0000 aopalliance
drwxr-xr-x 3 circleci circleci 4096 2018-12-12 19:25:38.388532199 +0000 commons-io
#2018-12-1219:29timgilbertSame results with ls -lRtc to get the ctime#2018-12-1219:42timgilbert...hmm, might be a PEBKAC problem#2018-12-1219:50timgilbertArgh, turns out it was a problem in my config where I changed the cache key things were being saved into but not the one where they were being restored from. Sorry for the noise! Once I fixed that up everything worked as expected and the deps are not being redownloaded in the second step.#2018-12-1219:52alexmilleryay#2018-12-1300:05Nolanhey everyone! have been loving tools.deps and have had no issues until i abruptly started receiving Error building classpath. with no apparent configuration change. has anyone seen this? i’ve been using deps in this way for a few weeks. anyone have any ideas on how to start debugging?#2018-12-1300:15Nolanfor the next person: ssh-add -l may reveal that the identity previously being used is no longer there. still not sure exactly why that happened#2018-12-1300:24alexmillermaybe your ssh-agent died and restarted?#2018-12-1301:38Nolanmust’ve. but it’s happily back at it now. i really appreciate what tools.deps has done for us. can’t say it enough—thank you for all your work on it.#2018-12-1300:28richiardiandreawhen I create clojure -Spom can I tell clojure not to include org.clojure/clojure? I don't have it in my project deps.edn#2018-12-1300:32hiredmanclojure merges in .clojure/deps.edn#2018-12-1300:32hiredman-Srepro Ignore the ~/.clojure/deps.edn config file#2018-12-1300:33richiardiandreauhm I have in the back my head a voice that says that that does not work...#2018-12-1300:33richiardiandreawill try it#2018-12-1300:34hiredmanit will likely break other things if you don't include a dependency on clojure somewhere#2018-12-1300:34richiardiandreaI am trying to build a pom.xml for ClojureScript so it should be fine#2018-12-1300:35richiardiandreayeah it still adds
<dependency>
<groupId>org.clojure</groupId>
<artifactId>clojure</artifactId>
<version>1.9.0</version>
</dependency>
#2018-12-1300:35hiredmanare you sure something else isn't puling it in?#2018-12-1300:36timgilbertI was just looking at this bit of code, I’m pretty sure -Spom will include your system-wide deps too, eg /usr/local/lib/clojure/deps.edn or whatever it’s called#2018-12-1300:37richiardiandreawell, I run clojure -Srepro -Spom after having deleted it...my deps.edn seems clean...dunno...#2018-12-1300:38timgilbertOn the subject of pom generation, I’ve just published garamond, which rewrites the pom.xml that -Spom generates with a version ID from git tags: https://github.com/workframers/garamond#2018-12-1300:39richiardiandreamaaaan that's awesome I was fiddling with scripts 😄#2018-12-1300:40timgilbertI still have a few features I’m working on, but it’s ready for use. It uses itself plus juxt/pack and deps-deploy to publish itself to clojars sans a project.clj#2018-12-1300:40richiardiandreausing those as well at the moment :+1:#2018-12-1300:41richiardiandreathank you so much, this will definitely simplify my scripts!#2018-12-1300:41hiredmancheckout clojure -Srepro -Sdescribe#2018-12-1300:41timgilbertAwesome#2018-12-1300:42richiardiandreaoh /home/arichiardi/.local/lib/clojure/deps.edn is something I haven't created#2018-12-1300:42richiardiandreaand indeed it contains the clojure version above#2018-12-1300:42hiredmanyeah#2018-12-1300:42richiardiandreatricky trick! thank you! 😄#2018-12-1300:42hiredmanI think the install scripts create that with the idea that it is the "system" level deps.edn#2018-12-1300:42richiardiandreacan I just get rid of :deps in there?#2018-12-1300:42hiredmanI don't see why not#2018-12-1300:43richiardiandrealemme do that then 😄#2018-12-1300:43hiredmanor just delete the file#2018-12-1300:44richiardiandreayeah that's right it works!#2018-12-1300:46richiardiandreathank you very much @hiredman#2018-12-1301:20alexmillernote that installing / upgrading clj will rewrite that file#2018-12-1301:33seancorfieldDeleting the "system" level deps.edn will remove the default Maven repos and a few other things, right?#2018-12-1301:34seancorfieldYeah, :mvn/repos and :paths (which defaults to src)...#2018-12-1307:46borkdudeis tools.deps officially supported on Windows as well? I didn’t think about this when choosing it as the thing for Advent of CLJC but I didn’t hear any complaints yet#2018-12-1309:13dominicmNot yet it isn't#2018-12-1310:36borkdudeI hear a lot of people are using WSL on Windows now#2018-12-1313:02seancorfieldYup I do nearly all my clojure development on windows via wsl because of that. #2018-12-1316:15normanIs there a reliable non-brew way to get clj on OS X?#2018-12-1316:26alexmillerwe are only providing brew as a means of install, but you could basically use the linux instructions too I think - they’re both basically doing the same thing#2018-12-1316:26alexmillerit’s essentially just putting files in the right places#2018-12-1318:21delaguardoHi there. I have some problems with pulling git deps from private repo. My ssh-agent is configured and running but I'm getting USERAUTH fail every time. Is there some corner cases where it might fail that I must be aware of?#2018-12-1318:28seancorfield@delaguardo Using https? If so, that's a known limitation of tools.deps. This is the uber-ticket for possibly solving that https://dev.clojure.org/jira/browse/TDEPS-91#2018-12-1318:28seancorfieldNote that it links to three related git/auth tickets.#2018-12-1318:28delaguardoNo, I'm using /cdn-cgi/l/email-protection#2018-12-1318:29alexmillerit’s a star-shaped room, so mostly corners#2018-12-1318:29alexmillerso if you do ssh-add -l - do you see your identity?#2018-12-1318:30delaguardoYes, and I can clone that repo. But for some reason jgit throwing an exception#2018-12-1318:30alexmillerother common issues are having stuff you don’t need in ~/.ssh/config#2018-12-1318:30alexmillerand issues with host key types#2018-12-1318:30alexmillerthe former being more common in my experience#2018-12-1318:31alexmillerif you can dump your config here, I can eyeball it (can’t remember off-hand the offending things)#2018-12-1318:31alexmillerbut I would know it if I saw it#2018-12-1318:32delaguardoHm.. everything is working on another laptop. Only one difference - my key has passphrase and it is added to agent via osx keychain#2018-12-1318:33alexmillerthat should be ok. can you dump your config file?#2018-12-1318:34alexmillerI think it’s IdentityFile that causes problem in your config (and you should delete that line)#2018-12-1318:34alexmillerso something like this should be sufficient:#2018-12-1318:34alexmillerHost *
AddKeysToAgent yes
UseKeychain yes
#2018-12-1318:39delaguardoWill try to do that. Thanks!#2018-12-1318:40alexmillerthe macos version you’re using on the two machines can also make a difference - they have changed the keychain integration a couple times#2018-12-1322:26manderson@alexmiller RE: the AUTHFAIL issue, I added a patch to https://dev.clojure.org/jira/browse/TDEPS-49 that fixes the problem without having to remove the IdentityFile line in your config. Not sure if you saw it as it was just before Conj, but wanted to mention it.#2018-12-1322:39alexmillerDidn’t see it, thx#2018-12-1406:22didibusSomeone brought up that tools.deps can't transitively resolve project.clj dependencies. Is that true?#2018-12-1406:28orestisAFAIK, yes. #2018-12-1406:32didibusOh, so until all lib upgrade, it could cause some issues there#2018-12-1406:32didibusto add deps.edn#2018-12-1406:33didibusCan it handle pom.xml ?#2018-12-1406:33didibusOr only deps.edn ?#2018-12-1407:00orestisI think Pom is fine, but not 100% sure. I believe Pom is used for maven-things so presumably it works fine. #2018-12-1413:47alexmillerThat’s correct#2018-12-1407:00orestisA usual trick is to fork a git library, at a deps.edn and use your fork with updated dependencies until it’s fixed upstream. #2018-12-1407:36dominicmJust to be clear, this only affects git dependencies. Nobody would depend on a git repo with only a project.clj.#2018-12-1407:38dominicmIt is absolutely fine if you depend on a git repo, which depends on a leiningen-built project on clojars or maven central.#2018-12-1723:01martinklepschWhen passing some serialized EDN to clj is that when people use commas instead of spaces to get it regarded as a single argument?#2018-12-1723:01martinklepschclj -m some.ns '{:my "data"}'
#2018-12-1723:14dominicmYeah. But not for the above case. #2018-12-1723:14dominicmIt's inside of deps.edn files.#2018-12-1723:42aaelonyHi - longtime leiningen user here. I’m on a mac, I am using home brew, but the clj that gets installed stacktraces miserably#2018-12-1723:42aaelonyHere is the stacktrace: $ clj -Sdescribe
Error building classpath. EOF while reading
java.lang.RuntimeException: EOF while reading
at clojure.lang.Util.runtimeException(Util.java:221)
at clojure.lang.EdnReader.readDelimitedList(EdnReader.java:746)
at clojure.lang.EdnReader$MapReader.invoke(EdnReader.java:680)
at clojure.lang.EdnReader.read(EdnReader.java:145)
at clojure.lang.EdnReader.read(EdnReader.java:111)
at clojure.edn$read.invokeStatic(edn.clj:35)
at clojure.edn$read.invoke(edn.clj:14)
at $slurp_edn.invokeStatic(io.clj:29)
at $slurp_edn.invoke(io.clj:22)
at clojure.tools.deps.alpha.reader$slurp_edn_map.invokeStatic(reader.clj:38)
at clojure.tools.deps.alpha.reader$slurp_edn_map.invoke(reader.clj:35)
at clojure.tools.deps.alpha.reader$slurp_deps.invokeStatic(reader.clj:58)
at clojure.tools.deps.alpha.reader$slurp_deps.invoke(reader.clj:54)
at clojure.core$map$fn__5587.invoke(core.clj:2745)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.Cons.next(Cons.java:39)
at clojure.lang.RT.boundedLength(RT.java:1785)
at clojure.lang.RestFn.applyTo(RestFn.java:130)
at clojure.core$apply.invokeStatic(core.clj:659)
at clojure.core$apply.invoke(core.clj:652)
at clojure.tools.deps.alpha.reader$merge_deps.invokeStatic(reader.clj:73)
at clojure.tools.deps.alpha.reader$merge_deps.invoke(reader.clj:70)
at clojure.tools.deps.alpha.reader$read_deps.invokeStatic(reader.clj:78)
at clojure.tools.deps.alpha.reader$read_deps.invoke(reader.clj:75)
at clojure.tools.deps.alpha.script.make_classpath$combine_deps_files.invokeStatic(make_classpath.clj:47)
at clojure.tools.deps.alpha.script.make_classpath$combine_deps_files.invoke(make_classpath.clj:43)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:68)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
#2018-12-1723:42aaelonyany ideas welcome#2018-12-1723:42aaelony(or I’ll go back to leiningen#2018-12-1723:42aaelony)#2018-12-1723:43seancorfieldWhat does cat ~/.clojure/deps.edn show?#2018-12-1723:43dpsuttonwhen i run clj -Sdescribe it gives me some output that includes config-files ["/usr/local/lib/clojure/deps.edn" "/home/dan/.clojure/deps.edn" ]. There are some deps.edn files and it seems like one of these is broken. can you check your equivalent ones#2018-12-1723:43aaelonyThat’s likely the culprit… $ cat ~/.clojure/deps.edn
;; The deps.edn file describes the information needed to build a classpath.
;;
;; When using the `clojure` or `clj` script, there are several deps.edn files
;; that are combined:
;; - install-level
;; - user level (this file)
;; - project level (current directory when invoked)
;;
;; For all attributes other than :paths, these config files are merged left to right.
;; Only the last :paths is kept and others are dropped.
{
{:aliases
{:new {:extra-deps {seancorfield/clj-new
{:mvn/version "0.5.5"}}
:main-opts ["-m" "clj-new.create"]}
}
;; Paths
;; Directories in the current project to include in the classpath
;; :paths ["src"]
;; External dependencies
;; :deps {
;; org.clojure/clojure {:mvn/version "1.9.0"}
;; }
;; Aliases
;; resolve-deps aliases (-R) affect dependency resolution, options:
;; :extra-deps - specifies extra deps to add to :deps
;; :override-deps - specifies a coordinate to use instead of that in :deps
;; :default-deps - specifies a coordinate to use for a lib if one isn't found
;; make-classpath aliases (-C) affect the classpath generation, options:
;; :extra-paths - vector of additional paths to add to the classpath
;; :classpath-overrides - map of lib to path that overrides the result of resolving deps
;; :aliases {
;; :deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.5.460"}}}
;; :test {:extra-paths ["test"]}
;; }
;; Provider attributes
;; :mvn/repos {
;; "central" {:url ""}
;; "clojars" {:url ""}
;; }
} #2018-12-1723:44hiredmanwhat does clojure -Srepro -Sdescribe show?#2018-12-1723:44dpsuttonit looks like aliases is malformed#2018-12-1723:44aaelonyyeah#2018-12-1723:44dpsutton@hiredman s/he can't get any output due to this being unbalanced#2018-12-1723:44seancorfieldThat's where your EOF is coming from @aaelony -- you're missing a }#2018-12-1723:45hiredman@dpsutton right, clojure -Sdescribe would error if either the system or user level deps.edn had a problem, clojure -Srepro -Sdescribe would error if it was the system level#2018-12-1723:45aaelonyCan I nuke the ~/.clojure/deps.edn and try again?#2018-12-1723:45seancorfield{:aliases
{:new {:extra-deps {seancorfield/clj-new
{:mvn/version "0.5.5"}} ; NEED EXTRA } HERE
:main-opts ["-m" "clj-new.create"]}
}#2018-12-1723:45dpsuttonah thanks for explaining @hiredman#2018-12-1723:46aaelonygot it - that is now fixed#2018-12-1723:46aaelonythanks#2018-12-1723:48aaelonyyay - all set now… phfew.#2018-12-1723:48aaelonybig thanks all around#2018-12-1723:50seancorfieldHappy to help! At work we drive everything with clojure CLI -- we started with Leiningen back in 2011, switched to Boot at the end of 2015, and switched completely to the new CLI stuff this year.#2018-12-1723:51aaelonyGreat - I’ve been mostly using R for the past year++ and getting back into Clojure and updating my toolset from lein…#2018-12-1723:52seancorfieldCool. And you can get started with Clojure 1.10 since the "gold" release of that dropped today which greatly improves error reporting in a lot of cases!#2018-12-1723:53aaelonyyeah - ironically, I didn’t feel the error reporting pain (until just now). The clj tool is still on 1.9 until it gets released to home brew.#2018-12-1723:56alexmillerThis particular error isn’t going to be any different with 1.10 afaik #2018-12-1723:57aaelonyfor me, this was a “not knowing what you don’t know” kind of error#2018-12-1723:57alexmillerThe error reporting differences are primarily in the repl and this is just throwing an exception out of the main#2018-12-1723:58aaelonyperhaps if there was a spec check on the ~/.clojure/deps.edn file#2018-12-1723:58alexmillerIt would probably be reasonable to better trap around config file reading and give a better semantic error here#2018-12-1723:59alexmillerWell, we’ve been supporting back to Clojure 1.8 in tools.deps, which is pre spec#2018-12-1723:59alexmillerI will probably bump that up soon#2018-12-1723:59aaelonyI see#2018-12-1723:59alexmillerBut in this case, it didn’t even read so would never be able to do a spec check anyhow#2018-12-1723:59aaelonyThough I read about it in the docs, I was pretty much unaware that the ~/.clojure/deps.edn file was involved#2018-12-1800:00aaelonyI should have known better#2018-12-1800:01alexmillerError could be more informative for sure#2018-12-1800:01seancorfieldhttps://github.com/seancorfield/dot-clojure updated now that 1.10 has dropped: (! 528)-> clj -A:master
Clojure 1.11.0-master-SNAPSHOT
user=>
🙂#2018-12-1800:02aaelony🙂#2018-12-1820:16kennyDoes :deps/root work for both local and git deps?#2018-12-1820:21kennyIt appears it doesn't.#2018-12-1821:12dominicmWhich doesn't it work for? It doesn't make sense for local dependencies really. #2018-12-1821:51alexmillershould work for both#2018-12-1821:51alexmillerin the case of a local dep it’s a directory relative to :local/root#2018-12-1821:57kennyIt doesn't appear to work for local/root. I get the following message:
Error resolving /home/kenny/compute_software/app-ions: Manifest type not detected when finding deps for compute/alerts-api in coordinate {:local/root "../alerts", :deps/root "api", :sha "ca8f741dd264fa13a900cb1d283e32a21f4c21c8"}
#2018-12-1821:58kennyIf I change :local/root to ../alerts/api it works as expected.#2018-12-1822:03alexmillerthe message is saying that it’s looking for a manifest file (deps.edn or pom.xml) at the specified location and not finding one#2018-12-1822:04dominicmWhich location is it looking in? Does it consider deps/root when searching? #2018-12-1822:04alexmillerit’s looking at <:local/root>/<:deps/root>#2018-12-1822:05alexmillerso it should be looking for ../alerts/api/deps.edn here#2018-12-1822:05dominicmThat wouldn't explain it working with local/root alone? #2018-12-1822:06alexmillerwell, the question is where .. is relative to#2018-12-1822:06alexmillercurrently, it’s always relative to the current directory of wherever you are running#2018-12-1822:06alexmillernot the relative directory of whatever is depending on it#2018-12-1822:06alexmillerthis is a known issue with local roots and relative dirs#2018-12-1822:07alexmillerI feel like I’m missing important parts of the problem description here so just kind of guessing#2018-12-1822:09kenny#2018-12-1822:12kennyActually that's not correct.
pwd
/home/kenny/compute_software/app-ions
clj -A:dev -Spath
Error building classpath. Manifest type not detected when finding deps for compute/alerts-api in coordinate {:local/root "../alerts", :deps/root "api", :sha "ca8f741dd264fa13a900cb1d283e32a21f4c21c8"}
#2018-12-1822:13kenny> so it should be looking for ../alerts/api/deps.edn here
And that file exists:
ls ../alerts/api/deps.edn
../alerts/api/deps.edn
#2018-12-1822:21hiredmanI am not sure, but I suspect that coordinate results in it looking for /home/kenny/compute_software/app-ions/api/../alerts/#2018-12-1822:25hiredmanno I am not right, just confused#2018-12-1822:35hiredmanunless I am confused again https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions/local.clj#L27-L31 would need to include :deps/root in path (instead of the empty string) in order to detect the deps.edn in :deps/root#2018-12-1822:57alexmilleryou’re not using that function#2018-12-1823:03hiredmanoh, yep, further confusion, that looks like it does that kind of resolution, but then doesn't appear to actually be called anywhere in tools.deps#2018-12-1823:05alexmillerthe manifest is being found in https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions/local.clj#L33-L38 which will fall into the last case and call https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions.clj#L21-L29#2018-12-1823:06alexmillerbut I will actually go back and say it’s not combining the two paths - it’s really just using :local/root#2018-12-1823:06alexmillerI was misremembering the code that’s in the git extension#2018-12-1917:20seancorfieldUpgraded clojure to the 1.10 release via brew and then wondered why my outdated dependencies checker was no longer working... and then realized that TDEPS-82 is not yet fixed!#2018-12-1918:13alexmillerI am working through some tdeps stuff right now…#2018-12-1918:13alexmillerwill try to hit that one#2018-12-1918:14alexmillerplanning to get to a release by end of week#2018-12-1918:25seancorfieldThank you!#2018-12-2022:56geraldodevIt would be nice if clj show invalid edn parsing errors#2018-12-2023:00dominicmI think there is a ticket for that, it's been mentioned. I think one would be welcome if you can't find one, if you can, a vote helps prioritize. #2018-12-2023:06alexmillerI was planning to look at that for the next release#2018-12-2023:06alexmillerbut there are limits to what we can do other than say “the edn is invalid”#2018-12-2023:07alexmillersince the edn file is a map, it either will read the whole thing, or it will fail (if you miss a brace for example)#2018-12-2023:07dpsuttonand bang for the buck i suppose. EOF encountered while ... is the current message, right?#2018-12-2023:07alexmilleryeah#2018-12-2023:08dpsuttonit seemed straight forward but maybe that is stockholm syndrome#2018-12-2023:08dominicm@alexmiller is this the same behaviour as for Clojure? Could we look at something like the figwheel error tool? #2018-12-2023:08alexmillerI’m planning to catch this case and make the error message better (and skip the stack trace) but it’s probably not going to actually help you in a lot of cases#2018-12-2023:08dominicmProbably good questions for JIRA 😊 #2018-12-2023:08alexmillersorry, don’t have context on figwheel#2018-12-2023:09dominicmI think figwheel can print out a snippet and give you an ASCII arrow pointing at the error. #2018-12-2023:09alexmillerthe errors I see reported are almost always invalid edn (usually a missing brace), not a valid map that is invalid#2018-12-2023:10alexmillerI think figwheel is doing that with spec-like validation on invalid values, but that’s not the error I’m describing#2018-12-2023:10alexmillerwe have specs and (if we bump the min clojure version) can check the read map against the specs to get that, but again, that’s not where I see most issues#2018-12-2023:11geraldodevI was bitten by Map literal must contain an even number of forms. Even on a modest file size it's very pratical to be pointed to the exact spot. I've downloaded diogo149/lein-edn-validator to help me find .#2018-12-2023:13geraldodev@dominicm tons of issues. ppl are using tools.dep. That's good!#2018-12-2023:13dominicm@alexmiller figwheel also does something with the data coming out of cljs compiler, and can information related to this.
I suppose in theory tdeps can be even smarter if the data was available from clojure.edn#2018-12-2023:24dominicmhttps://github.com/diogo149/lein-edn-validator/blob/master/src/leiningen/edn_validator.clj interesting approach.#2018-12-2023:56alexmillerWell, I’m not planning to do that#2018-12-2108:06dominicm@alexmiller out of curiosity, would you accept it as a contribution via a third party library or similar? Assuming startup time didn't hurt too much. #2018-12-2201:42dominicmIs there any reason :default-deps shouldn't be available as a top-level key? My reasoning being that I actually want any transitive dependencies that are newer than what I specify to be used instead. Maybe default-deps doesn't quite fill that niche. But if I depend on clojure 1.8, and cognitect-aws depends on clojure 1.10, I'm happy for 1.10 to be used. Maybe a "linter" could inform on this 🤔#2018-12-2221:16borkdudehow does one force download of dependencies in a deps.edn project?#2018-12-2221:36ghadi-Sforce#2018-12-2222:11borkdudeI figured, but that also opens a repl …
$ clojure -Sforce
Clojure 1.10.0
user=> ^D
#2018-12-2222:12borkdudealso, that forces recomputation of the classpath, I only want to force the initial downloading of stuff, else return fast without doing anything extra#2018-12-2222:13borkdudeit seems the problem solved itself on circleci, just do stuff and it will take care of itself 😉#2018-12-2222:22ghadi-Sforce -Spath @borkdude #2018-12-2222:22ghadiThat will just printout the computed classpath#2018-12-2222:25borkdudethanks#2018-12-2916:28borkdudeis there something to update a deps.edn file to the latest git sha’s ?#2018-12-2916:29borkdudecool: https://github.com/Olical/depot#2019-12-3115:51jmvhi all, do deps.edn git refs support repos with multiple subprojects? i'm wondering if i can specify a path into the repo to find the deps.edn file along with the sha#2019-12-3118:06richiardiandrea@jmv we use :local/root for that, not sure about sha#2019-12-3118:12seancorfield@jmv305 :deps/root should let you navigate into a subproject within a git repo.#2019-12-3118:50jmvthank you, i will give that a try!#2019-01-0221:33alexmillertools.deps.alpha 0.6.474 and clj 1.10.0.408 are now available.
Changes:
* TDEPS-79, TDEPS-109 Fix infinite loop in resolving recursive deps tree (will often improve resolution perf in other cases as well)
* TDEPS-50, TDEPS-109 Fix handling of Maven classifiers during resolution
To explicitly specify a Maven classifier in a dep, specify it in the lib name as follows: groupId/artifactId$classifier
Note: the :classifier attribute in Maven dep coordinates is no longer supported and will now produce an error.
* TDEPS-107 Canonicalize exclusion groupIds in pom gen
* TDEPS-82 Ensure -Sdescribe doesn't trigger resolution
* Error handling improvements when Maven artifact is not found
* Error handling improvements for invalid syntax in deps.edn
#2019-01-0221:34seancorfieldTDEPS-82 ❤️ ❤️ ❤️#2019-01-0307:39ChrisIs it necessary to reinstall from http://clojure.org to upgrade to these releases?#2019-01-0313:31alexmillerYes? How else would you get them?#2019-01-0314:55borkdudesome packages have an upgrade command (but this would still get the updates from http://clojure.org)#2019-01-0318:24ChrisYes, I was wondering if there was an update command or it even automatically upgraded the tools deps lib.#2019-01-0311:49geraldodevI tried the last tools.deps and I have a fulcro depified project that has nubank/workspaces {:mvn/version "1.0.2" :exclusions [com.cognitect/transit-java]}
With the new version it does not loads transit-java. The problem is that it is excluded from workspace but its a dependency of transit-clj (of fulcro) and shadow-cljs . It looks like the exclusion is for good (affect other packages). clj -Stree does not show transit-java. When I revert to 1.9.0.397 the error is gone#2019-01-0313:31alexmillerCan you be more explicit with a repro to make sure I understand?#2019-01-0314:11alexmiller@geraldodev ^^#2019-01-0314:11alexmillerin particular, would love to have your full deps.edn#2019-01-0316:11geraldodevMy deps.edn, good_tree.txt was produced with linux-install-1.9.0.397.sh
bad_tree.txt was produced with linux-install-1.10.0.408.sh. Look that there are no transit-java with the latter version. It was supposed to be there because it is a dependency of fulcro and shadow-cljs#2019-01-0316:13geraldodevThe only deps missing is a variation of sean corfield's ~/.clojure/deps.edn, but I think it does not play any role on this problem#2019-01-0316:23alexmillerwhat is the command you’re running to get the trees?#2019-01-0316:23alexmillerjust clj -Stree ? or using any aliases?#2019-01-0316:27geraldodevclj -Stree on the project directory#2019-01-0316:27geraldodevno aliases.#2019-01-0316:28geraldodevThe resolution appears to be different on the most recent version#2019-01-0316:28alexmillerthx, I will take a look#2019-01-0316:28geraldodevnp, thnx for your work on tools.deps#2019-01-0317:23alexmiller@geraldodev that’s an interesting case. there are basically multiple dependency paths to transit-clj 0.8.313 (all same version). The path in which it is first found and selected (nubank/workspaces -> com.cognitect/transit-clj) happens to exclude its transitive dep, com.cognitect/transit-java.#2019-01-0317:24geraldodevI checked the version. It was strange to me because it is the same like you saw#2019-01-0317:25alexmillerin simplest form:#2019-01-0317:25alexmillertree:
A (excl C) -> B -> C
D -> B -> C
selected:
A, D, B
#2019-01-0317:26alexmillerwhereas if B happened to be encountered via D first, C would be included#2019-01-0317:27alexmillerI know why what you’re seeing happened to change, but I think this problem likely existed on the prior version too, but via some other arbitrary ordering#2019-01-0317:28alexmillerI will need to think about this a bit#2019-01-0317:28geraldodevBut you are ble to get transit-java on prior version#2019-01-0317:28alexmilleryou happened to get transit-java on prior version#2019-01-0317:29geraldodevvia arbitrary ordering. I get it. It's a coincidence...#2019-01-0317:29alexmillerbut I don’t actually think you’re guaranteed that - it’s probably a result of the order the deps are being encountered#2019-01-0317:29alexmillerI’m not certain of that, but that’s my suspicion#2019-01-0317:30alexmillerI think what this dependency tree is saying is not self-consistent#2019-01-0317:31alexmillerthat is, if you treated the deps as logical propositions, there is no solution that satisfies both excluding C under A and including it under D in that simple version above#2019-01-0317:32alexmillergiven that it’s the same B in both cases#2019-01-0317:33alexmillerfor example, if you run this same example in Maven, you get a match of the “bad” tree (no transit-java included at all)#2019-01-0317:34timgilbertWouldn't you want it to be like:
tree:
A1 (excl C1) -> B1 -> C1
D2 -> B2 -> C2
selected:
A1, D2, (resolve-duplicate #{B1 B2}), C2
?#2019-01-0317:34alexmillerin this particular case, the versions of B and C in both branches are identical#2019-01-0317:35alexmillerso I omitted the versions#2019-01-0317:35alexmillerif the two B’s are identical, then it shouldn’t matter which one you choose#2019-01-0317:36alexmillerand exclusions are never version-specific#2019-01-0317:36geraldodevThere are no explicity dependency of transit-java on lein's fulcro template. It appears to include it.#2019-01-0317:37alexmillertree:
A1 (excl C) -> B1 -> C1
D1 -> B1 -> C1
selected:
A1, D1, B1
#2019-01-0317:38alexmilleris a version-specific example (and the exclusion intentionally does not have a version, as exclusions are not version-specific)#2019-01-0317:38timgilbertI guess I was thinking the semantics of exclusion were more "omit C from the entire transitive tree of A", not "globally exclude C from the tree"#2019-01-0317:39alexmillerit is the former#2019-01-0317:39alexmillerbut I’m saying that omit C1 can’t be consistent with also saying include D1 which depends on B1 which depends on C1#2019-01-0317:40alexmilleryou’re saying both omit C1 and include C1 - we can’t do both#2019-01-0317:40timgilbertI see the contradiction, yeah...#2019-01-0317:43timgilbertI guess I'd just intuitively expect that in order to get the behavior where C is excluded, you'd have to do
tree:
A1 (excl C) -> B1 -> C1
D1 (excl C) -> B1 -> C1
selected:
A1, D1, B1
#2019-01-0317:43alexmilleryes, that is what you have to do in either clj or Maven#2019-01-0317:44alexmillerthere have been requests for global exclusions as a feature but that’s not currently a thing#2019-01-0317:45alexmillerif anything were to be done on this, it would be to make it non-arbitrary#2019-01-0317:46alexmillerbascially if you have B and B (excl C), do not treat those as interchangeable - prefer one over the other#2019-01-0317:46alexmillerthis gets tricky when combined with version comparisons though#2019-01-0317:46alexmillerand I’m not sure which one you would want to prefer#2019-01-0317:48alexmillerseems like geraldodev’s expectation is that you would prefer including it#2019-01-0317:50geraldodevyes, like lein does#2019-01-0317:50alexmillerlein relies on Maven’s selection, which is also order-dependent#2019-01-0317:51alexmillerit wouldn’t surprise me if the ordering affected what you got in lein too#2019-01-0317:51geraldodevif workspace is not picking a version for you and another package is so be it. because it needs to be there. the version being the same is irrelevant to a greater proposal of having the lib#2019-01-0317:52alexmillerthis is not about version picking though, it’s about inclusion vs exclusion#2019-01-0317:52geraldodevbut the exclusion was caused because of the version being the same#2019-01-0317:55alexmillerthere is no set of dependencies that can satisfy the constraints stated, so the most correct answer here is actually to fail#2019-01-0317:55alexmillerwe’re just pouring semantics over the top of a logically inconsistent case#2019-01-0317:55alexmillerwe do that all the time when selecting deps though wrt versions#2019-01-0317:56hiredmanthe question is kind of, are exclusions global, or are exclusions local. are exclusions applied while determining what the transitive dependencies of a given dependency are, or are they applied later after all the dependencies have been determined an a graph has been created#2019-01-0317:56alexmillerthat’s not the question#2019-01-0317:56alexmillerexclusions are not global#2019-01-0317:56alexmillerthey are always contextual#2019-01-0317:58geraldodevlein deps :tree on template project after lein ancient :upgrade. It uses workspaces on 1.0.2#2019-01-0317:59alexmillerthe selection algorithm is attempting to satisfy (often contradictory) constraints while also balancing performance concerns#2019-01-0317:59geraldodevtransit-java is being picked by transitive dependency of shadow-cljs#2019-01-0317:59alexmillerclj is intentionally not using the same algorithm as either lein or Maven#2019-01-0318:00alexmillerthose both prefer versions that are closer to the top or first encountered at a sibling level#2019-01-0318:00alexmillerclj prefers the newest#2019-01-0318:00hiredmanmaybe I misstated it, but it sounds like the difference between the current observed behavior, and the behavior @geraldodev is suggesting is between applying exclusions before or after it has been determined that the B both A and D depend on is the same, which maybe isn't possible, but I described it as global because it seems like in order to know A and D depend on the same B you need a global view of the dependency graph#2019-01-0318:02geraldodev@alexmiller it's the newest and forbbidenn by one dependency#2019-01-0318:02alexmillerI think this discussion is missing too much context of what’s being considered to be correct#2019-01-0318:04alexmillerlike when choosing versions, you have to consider what versions you’ve seen, whether a version is under an exclusion of a selected version, etc#2019-01-0318:05geraldodev@alexmiller I'm in no place to argue about this. I just expected the lein behaviour.#2019-01-0318:05alexmillerand for performance reasons, we’re doing this in one pass, which limits information and when you know things#2019-01-0318:06alexmillerI’m still contending that the overall dependency tree here is logically inconsistent and there is no solution that meets all constraints#2019-01-0318:07alexmillerI would not be surprised if lein / Maven both have order-dependent answers to this and depending how you order things, you could see transit-java either being included or not here#2019-01-0318:08alexmillerif that’s not true, then their selection algorithm is effectively placing some preference, either intentionally or accidentally#2019-01-0318:08alexmillerclj’s selection algorithm is different, and may or may not make the same choice#2019-01-0318:09alexmillerI would prefer it not to be order-dependent though as that’s generally a source of confusion#2019-01-0318:09alexmillerMaven’s algorithm relies on order as means of selection, so it has to be#2019-01-0318:11hiredmansome of it is maybe ergonomic, having the exclusion listed on a particular dependency leads to the expectation (as incorrect as it may be) that the exclusion is somehow limited in scope to that dependency and won't effect dependency info from other deps#2019-01-0319:55kennyIn order to use Neanderthal on a Mac you need to specify these exclusions: https://github.com/uncomplicate/neanderthal/blob/7cf295119527843a6d1e9c17161b8a7b2f741309/examples/hello-world/project.clj#L6-L7. I cannot figure out a way to do this in a deps.edn. I tried this:
{:deps {uncomplicate/neanderthal {:mvn/version "0.19.0"
:exclusions [[org.jcuda/jcuda-natives :classifier "apple-x86_64"]
[org.jcuda/jcublas-natives :classifier "apple-x86_64"]]}}}
which results in
Error building classpath. Could not find artifact org.jcuda:jcuda-natives:jar:apple-x86_64:0.9.0 in central ()
I also tried
{:deps {uncomplicate/neanderthal {:mvn/version "0.19.0"
:exclusions [org.jcuda/jcuda-natives org.jcuda/jcublas-natives]}}}
which results in the same error. Is there a way to get this to work with a deps.edn?#2019-01-0320:24alexmillerexclusions in Maven can’t use classifiers afaik (certainly the deps.edn version does not support it)#2019-01-0320:28alexmillerI would expect something like this to work {:deps {uncomplicate/neanderthal {:mvn/version "0.19.0" :exclusions [org.jcuda/jcuda-natives org.jcuda/jcublas-natives]}}}#2019-01-0320:31kennyThat results in
Error building classpath. Could not find artifact org.jcuda:jcuda-natives:jar:apple-x86_64:0.9.0 in central ()
#2019-01-0320:31kennyHaving the exclusions that are in project.clj works as expected.#2019-01-0320:31kenny… in a project.clj.#2019-01-0320:31alexmilleryeah, the error there is accurate - there is no such jar#2019-01-0320:32alexmillerI guess this could be the exclusion code not being classifier aware#2019-01-0320:32alexmillerso it doesn’t know to try to exclude the os-dependent dependency#2019-01-0320:33alexmillercould I trouble you to file a ticket at https://dev.clojure.org/jira/browse/TDEPS ?#2019-01-0320:33alexmillerassuming that’s the issue, it shouldn’t be hard to fix#2019-01-0320:45alexmilleryeah, that was it. I have a fix for it#2019-01-0321:09kennyGreat! Should I still file a ticket?#2019-01-0321:38alexmilleryes please#2019-01-0321:38alexmilleror I can do it if you want#2019-01-0322:59kennyOk will do in a bit. Is there a workaround for now?#2019-01-0323:49alexmillerNo, don’t have one, sorry. I can rev a new version tomorrow with the fix though#2019-01-0401:10kennyThat'd be great. What would I need to do on my end to get that to work?#2019-01-0414:39alexmillerI filed a ticket at https://dev.clojure.org/jira/browse/TDEPS-112 and am working on a release. You’ll need to either “brew upgrade clojure” if on mac or re-install if on linux once it’s out (will take a few hours)#2019-01-0416:32kennySorry, meant to file the ticket - yesterday was quite busy. Thank you Alex!#2019-01-0416:58kennyI assume the brew upgrade will be available once this PR goes through? https://github.com/Homebrew/homebrew-core/pull/35692#2019-01-0417:41alexmilleryes#2019-01-0417:42alexmillerusually takes up to 24 hrs#2019-01-0722:49kenny@alexmiller I am still getting the same exception after running brew upgrade clojure.#2019-01-0722:53kennyActually I needed to open a new terminal. It appears to be working now.#2019-01-0803:19alexmiller:+1:#2019-01-0323:06kennyI am getting this error when using clojure on a different computer than my usual one.
Error resolving /Users/Mark1/Provisdom/math:
I can run git clone successfully. Why would the Clojure CLI not be picking this up?#2019-01-0323:49alexmillerIt uses jgit, so is a different pile of stuff#2019-01-0323:50alexmillerCan’t say I have enough to go on there#2019-01-0323:50alexmillerWhat os? Ssh?#2019-01-0323:50alexmillerPrivate or public repo?#2019-01-0323:53kennyMac OS. SSH. Public.#2019-01-0400:27alexmillerrun ssh-add -l to see if the ssh agent is running and has an identity in it (you should get probably 1 line back)#2019-01-0400:28alexmillergiven that it’s public, you could use the https url too, which would probably just work#2019-01-0400:29alexmillerif you have an ssh identity though, the likely problem areas are your /.ssh/config, or something with /.ssh/known_hosts or the host key types#2019-01-0401:10kennyI'll just switch the https for now and deal with it in the future if I need to access private repos.#2019-01-0410:56rickmoynihanFYI peeps there is a new version of lein-tools-deps that uses 0.6.474.
The main improvement is the TDEPS-82 fix which significantly improves runtime performance as we no longer resolve deps twice (Thanks Alex!). Also we no longer hardcode how we locate the clojure command line tool.
https://github.com/RickMoynihan/lein-tools-deps#2019-01-0412:52jeroenvandijkThank you!#2019-01-0413:11rcustodioDo you guys use depstar more or boot with this?#2019-01-0413:46jeroenvandijkI haven't used it on our major projects (leiningen, still too much hassle to switch). On new projects I'm working on boot and juxt.pack. expect for uberjar-ing boot works as expected#2019-01-0413:57rcustodioI see, good to know, thanks#2019-01-0417:05seancorfieldWe switched from Boot a few months ago and our entire dev/test/build pipeline is now based on clj/`deps.edn` and uses my fork of depstar (which has bug fixes and also the ability to create "thin" JAR files such as you'd push to Clojars).#2019-01-0417:11rcustodioIs that fork public?#2019-01-0419:31seancorfieldYes, and deployed to Clojars for easy usage. #2019-01-0419:32seancorfieldhttps://github.com/seancorfield/depstar#2019-01-0419:33rcustodioThanks!!#2019-01-0413:59rcustodioI’ve added javax.xml.bind/jaxb-api {:mvn/version "2.3.1"} but I’m having this Error building classpath. Could not find artifact javax.xml.bind:jaxb-api:jar:2.3.1 in central ()#2019-01-0413:59rcustodioCan anyone help me?#2019-01-0413:59rcustodio:deps {br.com.vikingmakt/augustus {:mvn/version "0.1.12"}
br.com.vikingmakt/njord {:mvn/version "0.3.0"}
cheshire {:mvn/version "5.8.1"}
clj-http {:mvn/version "3.9.1"}
clj-time {:mvn/version "0.15.0"}
com.novemberain/langohr {:mvn/version "5.0.0"}
com.novemberain/monger {:mvn/version "3.5.0"}
javax.xml.bind/jaxb-api {:mvn/version "2.3.1"}
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.1"}
org.clojure/clojure {:mvn/version "1.10.0"}
org.clojure/tools.logging {:mvn/version "0.4.1"}}
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2019-01-0414:00rcustodioNo idea what is wrong with it#2019-01-0414:04alexmillerthat leading quote on “2.3.1” looks weird - maybe a smart quote? “2.3.1"#2019-01-0414:04alexmillerI would retype that :)#2019-01-0414:05rcustodiowoooow…. thanks @alexmiller for some reason emacs was not showing any diff on quotes to me#2019-01-0414:06alexmillerwhen I copy pasted, it came over as quote-space in my terminal#2019-01-0414:06alexmiller¯\(ツ)/¯#2019-01-0414:06rcustodioI see.. next time I will try that, thanks#2019-01-0417:08orestisI’m using @seancorfield ‘s depstar and it works just fine. I might want to do AOT eventually but for now it does the job. #2019-01-0417:09seancorfieldAOT is evil. Stay away! 😇#2019-01-0417:11orestisI’ve seen you saying that before, but I’m not sure what the downsides are for things that go to production? Upside is less than half of the startup time (we went from 8s to 23s when switching from cambada to depstar). #2019-01-0419:28seancorfieldAOT of an entire app as the last step before building an uberjar for production deployment is fine if you care about startup time. #2019-01-0420:16orestisThanks!#2019-01-0417:11orestisI’d love to know what I’m missing :)#2019-01-0417:19aviI’m finding AOT useful for CLI tools#2019-01-0417:28ghadidepstar and AOT are not incompatible#2019-01-0418:39trevorCan I have a java (maven) project in github as a dependency with tools-deps?#2019-01-0418:45dominicmNo#2019-01-0418:45alexmillerno#2019-01-0418:45alexmillertools.deps / clj don’t do “builds” so there is no opportunity to compile the java source#2019-01-0418:45alexmilleryou will need to build and deploy the Java source as an artifact somewhere and depend on it that way#2019-01-0514:36mkvlrnot sure if that's something worth considering but the new $classifier syntax doesn't play nice with http://jitpack.io.#2019-01-0515:02alexmillerWhy? If there’s an issue, then please provide details or a ticket. I don’t get how it could affect jitpack#2019-01-0516:44mkvlrok, will do that Monday morning#2019-01-0514:46mkvlrand something seems not to be quite right yet for our use case (TDEPS-50) as we're seeing java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider. Not sure yet if it's related to the uberjar or what because it works in development. Will dig deeper on monday.#2019-01-0515:01alexmillerThat sounds like a native lib issue. There is a separate ticket to consider support around native libs - tools like lein do a bunch of extra stuff to make that work and I’m not sure yet if we need to take that on inside deps#2019-01-0517:55mkvlr@alexmiller oh sorry, please forget both things I've written above, it was entirely my fault: in our production build script I was explicitly requiring an old tools.deps version that didn't understand the $classifier syntax yet 🙈 Now that I've fixed that it's all working great 👌 Sorry again, thanks for your pointer and your great work. Enjoy the rest of your weekend!#2019-01-0606:13duncanmakwhere’s the source code to the clj tool?#2019-01-0606:32lilactownnot sure if it’s the source of truth, but I found the code here: https://github.com/clojure/brew-install/tree/1.10.0/src/main/resources#2019-01-0606:32lilactownclj calls clojure with rlwrap#2019-01-0609:02pesterhazy@duncanmak main code here https://github.com/clojure/tools.deps.alpha#2019-01-0609:03pesterhazywrapper scripts in the brew-install repo (not only for macOS but also Linux0#2019-01-0611:44alexmillerThose are the correct places#2019-01-0618:11duncanmakmorning @alexmiller#2019-01-0618:11duncanmaki’ve been clicking around, trying to understand how clj and tools.deps work#2019-01-0618:12duncanmakafaict, it’s all java code, so my question is, where / when does it change the classpath? does it start the jvm twice? once to calculate the right classpath, and once to actually run the instance?#2019-01-0618:13duncanmakand also, i’ve been looking for the code of the procurers#2019-01-0618:30alexmillerYes, it runs the jvm twice - once to compute the classpath then again to launch your program#2019-01-0618:30alexmillerCaching is used to avoid the first step when possible#2019-01-0618:31alexmillerThe procurer code is all under tools.deps.alpha.extensions#2019-01-0623:39duncanmak@alexmiller ahh, okay, so it’s not as magical as I thought, great!#2019-01-0623:39duncanmak@alexmiller where can i see it running twice, is that in the shell script wrapper?#2019-01-0623:39duncanmakhttps://github.com/clojure/brew-install/blob/1.10.0/src/main/resources/clojure#2019-01-0623:40duncanmakso I’m guessing, this is the first invocation, to resolve-tags: https://github.com/clojure/brew-install/blob/1.10.0/src/main/resources/clojure#L179#2019-01-0623:40duncanmakand then this is the 2nd and final run: https://github.com/clojure/brew-install/blob/1.10.0/src/main/resources/clojure#L346#2019-01-0700:07seancorfield@duncanmak Line 179 is only run if you ask clj to resolve tags for you. Optional. But required before building the class path from git repos.#2019-01-0700:08seancorfieldThis is where the classpath is built (unless it is already cached): https://github.com/clojure/brew-install/blob/1.10.0/src/main/resources/clojure#L301#2019-01-0700:10duncanmakCool#2019-01-0700:10duncanmakVery clever!#2019-01-0700:10duncanmakI'm working with Kawa Scheme, I'm hoping to build a similar thing for it also #2019-01-0715:10borkdudeit it me or is tools.deps missing from this question in the Clojure Survey?
22. Which tools do you use to compile/package/deploy/release your ClojureScript projects?#2019-01-0715:13alexmillerthat’s ClojureScript-specific - is that a thing?#2019-01-0715:14alexmillerI just added it, so it’s there if people want to pick it#2019-01-0715:15borkdudewell, I do a lot of things with cljs.main, which uses tools.deps?#2019-01-0715:15borkdudelike compiling. I deploy via scp.#2019-01-0715:15dpsuttonbe nice if cljs-test-display was added to the cljs testing question as well#2019-01-0715:19borkdudeso maybe cljs.main should be the option instead of tools.deps, dunno 🙂#2019-01-0715:26alexmillerI changed it to “clj / cljs.main”#2019-01-0715:26alexmillerwhat’s cljs-test-display?#2019-01-0715:27alexmillerthat seems like a visualization tool rather than the test runner?#2019-01-0715:30dpsuttoni suppose. then credit would go to figwheel and cljs.test for running the actual tests. but i feel like cljs-test-display is more than that. i just mention because that's my interface to all of our cljs tests and i was looking for it in the list#2019-01-0715:30dpsuttonand this is the worst room for this sorry 🙂#2019-01-0715:34alexmillerha, no worries - there is an Other category for stuff like this - often new stuff shows up there and then I add it the following year if it’s strong enough#2019-01-0720:14duncanmakhey @alexmiller, i also started looking into Groovy’s Grape, which I see to be something very similar to tools.deps for Clojure#2019-01-0720:15duncanmakis the choice of using Ivy vs Maven (or Eclipse Aether) significant? Or do the two APIs basically do the same thing and it’s just a personal choice to use one or the other?#2019-01-0721:00alexmillerI’ve spent some quality time with Grape in the past and worked on a build system that used Ivy extensively as well, so I’m somewhat familiar with both. I have never looked at Ivy’s actual code to see if it could satisfy what we needed wrt Maven repositories (although I’d be somewhat surprised if it didn’t ultimately use the same Maven APIs we are already using).#2019-01-0721:19alexmillerI glanced at it and it’s actually not using the Maven APIs at all afaict, so would be an alternate path to investigate for downloads at some future time (could also jettison both and just do the moral equivalent directly in Clojure, which might actually be better). We do use a bunch of Maven stuff to read pom files and we’d need that regardless though.#2019-01-0721:37kennyWhen starting a REPL with clj, is there a way to set default JVM opts?#2019-01-0721:39alexmillerno, but that is a ticket in the system for consideration#2019-01-0721:40alexmilleryou can pass them explicitly on the command line by prefixing with -J though like -J-Xmx1g#2019-01-0721:41kennyHmm ok. Any idea what the default -Xmx is?#2019-01-0721:41kennyOr maybe it doesn't set it at all? https://github.com/clojure/brew-install/blob/1.10.0/src/main/resources/clojure#L346#2019-01-0721:42alexmillerin latest JVMs there is no default - it’s computed as a fraction of system resources I think, or something like that#2019-01-0721:43alexmillerand more modern GCs can also release heap back to the OS#2019-01-0721:43alexmillerso no, we’re not setting any jvm settings - that’s up to you to determine#2019-01-0721:46kennyGot it, thanks.#2019-01-0721:51duncanmak@alexmiller thanks!#2019-01-0814:49johnI seem to be stuck on betas with brew#2019-01-0814:50johnI just ran brew upgrade clojure and it just updated with clojure 1.10.0.403 -> 1.10.0.411#2019-01-0814:50johnand
user=> *clojure-version*
{:major 1, :minor 10, :incremental 0, :qualifier "beta6"}
#2019-01-0814:52johnmy global deps.edn, duh :face_palm:#2019-01-0815:21borkdudeyeah, that’s what -Srepro is for 🙂#2019-01-0815:57alexmilleryeah, if you have a ~/.clojure/deps.edn, it won’t be modified (in case you’ve added your own config to it)#2019-01-0911:41gphilippLet’s say I have a repo named foo with 3 logical modules in their own subdirectory: a, b and c. Each have their own deps.edn with their required dependencies. I can also have a root deps.edn which refers to these modules using 3 {:local/root ...} deps. Now, I fail to see how I can add a dependency using the git dep scheme on say, module foo-b from brand new project bar which lives in a separate repo. Has anyone done something similar before ?#2019-01-0915:24gphilippWell, @matthias.margush just mentioned to me that actually we can add a :deps/root value to a git coordinate to specify a subrepo to use a subdirectory. That’s the answer to my issue above, but alas it’s not (yet) documented in https://clojure.org/reference/deps_and_cli or https://clojure.org/guides/deps_and_cli.#2019-01-0915:27alexmillerthere’s an issue to doc it at https://github.com/clojure/clojure-site/issues/307, just haven’t gotten to it#2019-01-1010:55gphilippThx @alexmiller. While I’m at it, we’ve been bitten by https://dev.clojure.org/jira/browse/TDEPS-74 (deps transitivity issue involving :local/root). Currently the scenario whereby project foo depends on module a of project bar using bar {:git/url "" :deps/root "a"} and bar.a depends on siblings module bar.b using {b {:local/root "b"}} doesn’t work. @matthias.margush has put together a patch which solves the issue (more info on the ticket), if you have some time to look at it that would be great. We’re not far from being able to switch over to tools.deps for several multi-modules projects.#2019-01-1011:41alexmilleryeah, there are several issues with the path stuff and I don’t think the change there is sufficient to handle the real scope of the issues#2019-01-1015:49matthiasThat makes sense - it did seem like the way coordinate specifiers interact could use some hammock time overall.#2019-01-1015:49matthiasWhat are the other jiras that relate?#2019-01-1015:52alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-106#2019-01-1015:52alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-96#2019-01-1015:52alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-52#2019-01-1015:53alexmillersome of those intersect with other problem areas (config file management)#2019-01-1418:59matthiasHi Alex. Reading through these in more detail, it looks fixes could potentially be bundled together with a fix for TDEPS-74... they are closely related, right? If it would be helpful at all, I'd be willing to take a look at these in context of the patch I started for TDEPS-74.#2019-01-1015:53alexmillerI’d say it’s unlikely I’m going to look at this set of things this week#2019-01-1016:28matthiasOh yeah, not expected. Just want to get it on the radar. tools.deps really opens up the ways that we can organize work across teams, and thanks for the work you all are doing on it.#2019-01-1015:55matthiasOne thing I noticed personally was that transport, location and manifest type are different things that relate and interact, but don’t map conceptually one to one to how extensions are dispatched.#2019-01-1016:00alexmillertransport and location aren’t uniform dimensions for procurers#2019-01-1016:01alexmillermanifest types are something that extensions dispatch on#2019-01-1016:23matthiasAre procurers things like maven, local, and git (and not pom, deps)?#2019-01-1016:55alexmillerYeah - there are two layers of multimethods#2019-01-1017:36matthiasI see.#2019-01-1017:43matthiasI think possibly related to your comment about uniformity of dimensions - a capability I’d like to be able to support for our engineers is to checkout a git/deps dependency locally while working on a cross cutting feature. External tooling could support this well enough for example by transforming a git coordinate to a local one in the local work area.
But the differences in the set (& maybe the semantics) of dimensions make the creation of tooling more bespoke than needed. I think specifically in this use case it would help for the :deps/root to work the same between git coordinates and local coordinates. (Unless that’s already the case and I misread the code.)
So such a tool needs to know the dimensions available and how they are used for each kind of coord.#2019-01-1023:40duncanmak@alexmiller i’m interested in learning more about how to download jars without using maven/ivy - do you know of anything that talks about how that works?#2019-01-1023:52seancorfield@duncanmak If you know the URL of the JAR in the repository, you can just download it via any means -- such as curl at the command line, or programmatically using URI and byte input streams etc.#2019-01-1023:53seancorfield(unless the repo is secured and then you'll need to factor in auth credentials)#2019-01-1100:01hiredmanmaven has different plugins for talking to repos, but the most common transport for talking to repos is http, so a maven repo can just be directory containing files on a webserver#2019-01-1100:02hiredmanthe directory tree structure in that case is actually the same structure as ~/.m2, so you can upload your m2 to s3, make it publicly accessible, then use it as a maven repo#2019-01-1100:03hiredmanthe complicated part of maven is parsing and understanding pom files#2019-01-1100:04hiredman(for example this is the clojars maven repo https://clojars.org/repo/)#2019-01-1100:12duncanmaki see#2019-01-1112:18eval2020Is dependency resolution in any way influenced by the contents of /.m2? Ie I’m considering caching /.m2 on CI but was wondering whether CI-runs without a local repository might actually yield helpful early warnings/errors (eg library updates that cause problems).#2019-01-1112:24jeroenvandijkNot caching would tell you if a dependency suddenly disappeared (which shouldn't happen for at least clojars), but otherwise I cannot think of a reason to do it. If the library is present locally it will not do an additional fetch unless you use -SNAPSHOT versions, which is probably not desirable anyway. Having said that i'm curious for other answers to this question#2019-01-1112:34eval2020that’s indeed what I would expect. I see there’s also .cpcache/ which seems to contain the actual result of resolution.#2019-01-1113:49jeroenvandijkActually now I come to think about git deps, where force pushes and deletion of repo's are a possibility. This might be a valid concern#2019-01-1113:50jeroenvandijkBut git deps don't really interact with ~/.m2 deps i think#2019-01-1115:29eval2020indeed - these end up in ~/.gitlibs and it seems you can’t set this path (as opposed to mvn/local-repo) so not easily cacheable on gitlab-ci.#2019-01-1116:05j0nidepending on your CI solution, you could add a full dependency pull to a step which depends on tests passing, so you don't waste time on it as part of your regular feedback cycle#2019-01-1113:38mike_ananev@alexmiller Hi. Is there any updates with cli tools for Windows? very expected feature#2019-01-1113:39alexmillerno update, sorry#2019-01-1115:45borkdude@mike1452 how is the WSL support in Windows nowadays?#2019-01-1116:56seancorfieldclj/`deps.edn` works really well on WSL. The problem you then get is editor integration of you use a windows editor with your repl running on Linux. #2019-01-1116:57seancorfield(You can't "load file" from windows because Linux won't understand the path the editor sends to it)#2019-01-1116:59seancorfieldThere are workarounds but it's a less than ideal process. I tend to select-all + evaluate (and I have symlinks set up in WSL so the file paths match windows, modulo the drive colon prefix). #2019-01-1120:51timgilbertSay, anyone executing their applications via clojure inside of docker images? Want to share any findings? I'm wondering how to handle caching (ie, whether to clojure -Sforce -Spath to download deps to the image, then cache them for faster startup)#2019-01-1121:23mike_ananev@borkdude corporate windows policies prohibits using WSL. so, half of our team, with BYOD devices, using clj cli tools, and another half with corporate PCs (Windows OS only) should use leiningen, which is available for Windows and allowed in our network . it is very inconvenient. we are still waiting for the windows version of Clj cli tools. Well, "First rule of Open Source, never ask when something will be done". And it is sad. 😞#2019-01-1121:44duncanmakhmm, interesting#2019-01-1121:44duncanmakwhat is making clj Unix-dependent?#2019-01-1121:46hiredmanit needs a powershell port I guess, which is an issue lein had a long for time, basically the people writing the tools don't use windows much and don't use powershell, so it is difficult for them to do the port or review it#2019-01-1121:53dottedmagI had to port a bunch of shell code to PS recently, and it was not so difficult, but annoying as hell. Favourite tidbits: 1) downloading a file with a progress indicator enabled is awfully slow (it looks like this indicator gets updated for every byte, and terminal on Windows is quite slow), and this progress indicator is enabled by default; 2) extracting a file from Zip using PS standard library cmdlet is ~10 times slower than directly invoking .NET code this cmdlet is wrapping.#2019-01-1121:55dottedmagOTOH one does not have to worry about differences between supported grep options or check if the tool is called shasum or sha256sum.#2019-01-1122:15alexmillerI have a not quite working port of clj to powershell. In the journal I just posted I asked people interested in actually working on dev to ping me here. If someone(s) are interested in really digging in, let me know and I will do what I can to guide.#2019-01-1303:24ajoberstar@alexmiller I've got some PowerShell experience and would be interested in helping this along. What still needs to be done?#2019-01-1203:53kenji_signifierHi, I was wondering if you have thought about adding clojure CLI to SDKMAN https://sdkman.io. I had a conversation with the main author Marco Vermeulen in https://gitter.im/sdkman/cli-dev and he needs a tarball that is usable by just untar. Leiningen added the support while ago. Plz check out https://github.com/technomancy/leiningen/issues/2204#2019-01-1204:29alexmillerone tricky thing in the clj script is that the script gets modified on install to point to the absolute location of the lib dir#2019-01-1204:29alexmillernot sure what the best way to handle that is#2019-01-1204:30alexmillerbut feel free to file an issue at https://dev.clojure.org/jira/browse/TDEPS#2019-01-1204:31alexmillerwould be happy to add it but I’m not going to have time to work on it, some whatever help you could provide would be useful#2019-01-1204:33alexmillerthe current install is basically a tar file plus an install script that mostly just moves files around, but also makes the jar location replacement mentioned above#2019-01-1206:22kenji_signifier@alexmiller Created https://dev.clojure.org/jira/browse/TDEPS-113 I’m happy to help this out. Should I enhance https://github.com/clojure/brew-install by creating a PR to add another package.sh for SDKMAN under script dir, or following the contribution workflow and sign the CA?#2019-01-1206:29alexmillerWe don’t do PRs in this repo so patch on the jira would be better#2019-01-1206:30alexmillerAnd ca is required#2019-01-1206:35seancorfieldSee https://clojure.org/community/contributing for the process @kenji_signifier if you haven't already seen it...#2019-01-1207:30kenji_signifierthx @alexmiller and @seancorfield I’ll follow the workflow once I create a patch for it.#2019-01-1308:59dottedmagIs there a way to have global :jvm-opts?#2019-01-1309:00dottedmagBesides making a shell wrapper around clj.#2019-01-1313:54alexmillerCurrently no, that’s an enhancement request#2019-01-1419:32borkdudesince there is no cognitect channel and this is a tools.deps project, I’ll post it here. there’s a trivial PR lingering for a while now. I wonder why there’s no interesting in accepting it from Cognitect Labs:
https://github.com/cognitect-labs/test-runner#2019-01-1419:33borkdudeI even forgot about the previous PR and made a duplicate one 🙂#2019-01-1419:36alexmillerSorry, Luke was minding that project and he’s not at Cognitect anymore#2019-01-1419:36alexmillerI’ll take a look when I get a chance#2019-01-1419:37borkdudecool 🙂#2019-01-1419:37borkdudeI’ve been using it more and more. There’s also a CLJS clone of it#2019-01-1419:40borkdudeghadi made the first PR. he is at cognitect right?#2019-01-1419:57dominicm@U050ECB92 is this any you became a space cat wizard? #2019-01-1419:48alexmillerHe is#2019-01-1419:51ghadimerged @borkdude#2019-01-1419:51ghadi209b64504cb3bd3b99ecfec7937b358a879f55c1 is master now#2019-01-1421:03aviAre there any examples out there of libraries that have both a project.clj and also a deps.edn, for compatibility with both Leiningen and tools.deps, but don’t duplicate the dependency specifications? I’m interested in possibly contributing such a feature to a library I use.#2019-01-1421:05dpsutton:plugins [[lein-tools-deps "0.4.1"]] is in one of our project.clj files. This allows the lein project to read deps out of the deps.edn file#2019-01-1516:06rickmoynihan:plugins [[lein-tools-deps "0.4.3"]] is the latest and should be a bit quicker at resolution.#2019-01-1520:27aviThanks!#2019-01-1521:04thosmosI've been using that for quite a while, and it works great. but now Cursive supports deps.edn directly so I mostly use that#2019-01-1609:52rickmoynihanlein-tools-deps is not without its issues… I should probably make them clearer on the repo.#2019-01-1613:52aviOnce I read the docs, I was up and running with no clear issues, at least not at first. Maybe some of the steps could be moved to the top of the readme, but no big deal#2019-01-1613:56rickmoynihanGood to know… but it can cause problems if you’re not aware of how it works… and it’s not made clear yet.
Essentially it currently flattens the whole transitive dependency graph. It does this to guarantee resolution is the same in both tools.deps and leiningen — but it does mean that downstream mvn resolution semantics can change.#2019-01-1613:57rickmoynihane.g wrt to :exclusions#2019-01-1614:45aviThat sentence seems very important… if it’s not in the readme already maybe just add it as-is?#2019-01-1614:50rickmoynihanI think it warrants a small section covering the implications tbh — will write it at some point.#2019-01-1614:51avi💪#2019-01-1614:51aviI’d be happy to review if that’d be helpful#2019-01-1614:53rickmoynihanyou’re welcome to comment on what is written — but it’ll probably be ok. Too busy to do it just now though.#2019-01-1614:53rickmoynihanthere are github issues that touch on it already#2019-01-1614:54avigot it!#2019-01-1421:05dpsuttonMakes it very simple to use lein for building/deploying but deps.edn for dev and running#2019-01-1421:39aviWill check it out, thank you!#2019-01-1421:42hiredmanI just pushed https://git.sr.ht/%7Ehiredman/dset/tree up over the weekend, it has both a deps.edn and a project.clj, with all the deps coming from deps.edn#2019-01-1500:14lilactownhow do you like using http://sr.ht?#2019-01-1500:21hiredmanso far all I have done is create that repo and push to it, seems fine, I appreciate that I can browse the site without letting it execute js#2019-01-1520:27aviGreat, thanks for sharing!#2019-01-1500:13richiardiandreawe are doing like the above, I have also a deps.edn only project but having lein makes it easy for colleagues to use Cursive, which still has some quirks with deps.edn#2019-01-1500:43cfleming@richiardiandrea It does? I’d like to hear about them.#2019-01-1500:44richiardiandreaah ah for sure 😄 one of my colleagues opened a couple of issues so you are aware of everything I think, don't remember which ones....wherever you see monorepo is us I guess!#2019-01-1521:46cflemingOk, I’ll check those out, thanks - AFAIK most things should be fixed these days.#2019-01-1515:16vemv👋
I asked this yesterday in off-topic, wondering what you think about it https://clojurians.slack.com/archives/C03RZGPG3/p1547483045535400
One thing I would fear if people widely adopted "SHAs for every dependency" is that two different libraries (let's call them A and B) could independently declare a dependency on C, but at different SHAs.
Is that a scenario that tools.deps can gracefully handle?#2019-01-1515:16alexmilleryes#2019-01-1515:16alexmillerwell depends what part you’re asking about#2019-01-1515:16alexmillerif you declare two sha’s in the transitive dep tree, it will use the newer one#2019-01-1515:21vemvthank you! I'll look into using tools.deps at work then 🙂#2019-01-1515:25alexmillerWhere newer = more descendant. If there is no descendant relationship (independent branches) then it’s an error#2019-01-1515:17alexmiller(same as we do for Maven artifacts)#2019-01-1520:37aviSo in this library I’m hoping to help out with, I’ve moved the dependencies from project.clj to deps.edn and added the plugin lein-tools-deps to project.clj. lein deps seems to work just fine, but when I run lein test I get:
$ lein test
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
Tests failed.
I tried changing the clojure version specified in deps.edn to 1.9.0 and 1.10.0 but that seemed to have no effect. Anyone know what’s happening here, what I’m missing?#2019-01-1520:49ghadiseems like your tests are not picking up the dependencies#2019-01-1520:50ghadiI wonder if your lein-tools-deps plugin isn't working in the test profile#2019-01-1520:57aviHmm, OK, I’ll look into that — thanks!#2019-01-1521:07aviAha, I needed to add :middleware and :lein-tools-deps/config to project.clj — as explained clearly in the docs of lein-tools-deps! (docs ftw)#2019-01-1521:07thosmos@aviflax I had that same problem with Cursive and the only thing I found to get around it was to downgrade lein-tools-deps to "0.4.1"#2019-01-1521:09aviHave you tried adding those keys to your project.clj ?#2019-01-1523:42aaelonyI have a codebase that I’ve instrumented with tools-deps and can run via clj -m something.main <args>
from the command line. What do folks typically do or what guidance is out there to make it easier for other (non clj) users to run it. In the past, I’d have done a lein uberjar and wrapped it in a bash script. Is there a better way?#2019-01-1600:05seancorfieldYou can still produce an uberjar and/or wrap it in a bash script https://github.com/seancorfield/depstar#2019-01-1603:14souenzzoAbout depstar/tools-deps jar'ing:
I need to deliver a jar that will run with java -jar my.jar. It's a client requirement.
There is some way to create it using tools-deps eco-system??
PS: without use things like clj -A:lein -m lein.uberjar#2019-01-1604:53richiardiandreaI am using https://github.com/juxt/pack.alpha but I am not exactly sure it does the same as leiningen#2019-01-1618:41dominicmPack works differently to leiningen, intentionally. #2019-01-1618:51seancorfieldI started to look at adding manifest support to depstar and realized I'd have to deal with AOT/`compile` unless you just set Main-Class: in the manifest to clojure.main... and I didn't have any enthusiasm for fixing that part 🙂 I may take another run at that at some point (it seems like it needs to create a temp folder, add it to the classpath, bind *compile-path* to that temp folder path, and run clojure.core/compile for the named main namespace … which just seems like a lot of work, just to enable java -jar and not need -m entry.point!).#2019-01-1618:55dominicm@seancorfield can depstar take a flag to add a folder? That's how pack makes AOT work #2019-01-1618:56seancorfielddepstar deliberately takes no options at the moment. It's "simple", rather than "easy" 🙂#2019-01-1700:31souenzzoMaybe someone will make this simple thing easy... 😜#2019-01-1601:26ajoberstar@alexmiller Still looking for community contributions on the Windows clj script? I'm interested and have PowerShell experience.
(Sorry for the dupe if you already saw my previous post and just haven't had a chance to reply.)#2019-01-1601:44alexmillerI saw it, thx, just been busy on other stuff#2019-01-1603:27seancorfield@souenzzo yes some of the tools.deps libraries add a manifest when building an uberjar. I had it in depstar at one point but preferred fewer options. #2019-01-1603:29seancorfieldOpen an issue and I'll put that code back in. Or you can try the other tools. I'm on my phone so I can only point you at the wiki in the tools.deps repo for a list. #2019-01-1606:19seancorfield(It would help if Issues were enabled for my project I guess… fixed!)#2019-01-1618:42dominicm@seancorfield any advice for running clj on WSL? Gotchas to be aware of? Preparing for a training course, I seem to have edge working, but I'm expecting real life issues. #2019-01-1618:44seancorfieldThe only annoying thing I've run into is that if you edit code on Windows, connected to a REPL of any sort running on WSL, any file-level ops you attempt will fail due to the mismatch of filepaths between the two environments. For example, a hot key to "load file" in your editor will generally send a full file path to the REPL, which in this case will start with C:\… when the WSL REPL is expecting /…#2019-01-1618:45seancorfieldMy workaround for that is select-all, eval-selection (but I may try to "fix" it in Chlorine directly).#2019-01-1618:46seancorfieldOh, caveat for that: I use a project repo path on Windows of something like C:\projects\myrepo and have /projects symlinks on WSL to /mnt/c/projects so the file paths are almost the same.#2019-01-1618:47seancorfield(that way, just dropping C: and using / instead of \ will work on both sides -- modulo the editor file path issue)#2019-01-1618:48seancorfieldAlso, since WSL is headless, you can't run REBL on it (so you end up having to hack that together with Leiningen on the Windows side.#2019-01-1618:48seancorfield(I haven't made any serious attempt to try to debug/workaround that)#2019-01-1618:49dominicmThat's a useful tip, thanks! #2019-01-1821:42Christian Johansenis there a special trick to deploying ClojureScript libraries to clojars with tools.deps? I tried doing clojure -Spom and then mvn deploy. I certainly did put a jar in clojars, but trying to use it I discover that it doesn't contain any sources 🤔#2019-01-1822:23eval2020@christian767 The pom needs build > resources. For a cljs-lib I use this pom: https://gitlab.com/eval/otarta/blob/master/pom.xml. Check contents of jar with jar tf foo.jar.#2019-01-1822:24Christian JohansenThanks!#2019-01-1822:24Christian JohansenSeems my mistake was not building the jar, D'OH!#2019-01-1822:24Christian Johansenthought it would be auto-built. and it was, but the jar maven built was useless, unsurprisingly, as it doesn't know about cljs files I guess#2019-01-1822:25Christian Johansenused mvn deploy:deploy-file with a prepackaged jar instead#2019-01-1914:19Christian JohansenI've had various minor issues figuring out how to combine tools.deps, figwheel-main, devcards, emacs/CIDER and Clojars, and got a lot of help in here, and now I've summarized my experience in a howto on configuring this combination: https://cjohansen.no/tools-deps-figwheel-main-devcards-emacs/#2019-01-1921:01richiardiandreathanks this is nice#2019-01-1921:02richiardiandreathe only difference I have there is deploying with deps-deploy...also sooner or later I would like to be able to sign the jar.#2019-01-1914:19Christian Johansenhope it can be of help to someone#2019-01-1921:00richiardiandrea@christian767 I have recently added deps.edn-only deploy in https://github.com/elasticpath/fonda - actually taking inspiration from @eval2020's work#2019-01-1921:03richiardiandreasorry I read I am late...too late 😄#2019-01-1921:28Christian JohansenYeah @slipset tipped me about deps-deploy on Twitter, I'm going to update the article to recommend it in place of maven :+1:#2019-01-1921:35slipset@richiardiandrea jar signing just landed on master for deps-deploy. Comments welcome.#2019-01-1921:36Christian Johansen👌#2019-01-1921:37slipsetHad to read a bit too much to figure out how to do that.#2019-01-1921:39slipsetThe lein code for that is somewhat b0rken (reading the gpg passphrase is a bit unstable at times), so I had to reimplement that…#2019-01-1921:39slipsethttps://github.com/slipset/deps-deploy/blob/master/src/deps_deploy/gpg.clj#L19#2019-01-1921:41Christian JohansenUgh, gpg. Can't think of anything that makes me feel more like a moron #2019-01-1921:42richiardiandreaWow timely! Thank you!#2019-01-1921:43slipsetInspired by @christian767’s blog 🙂#2019-01-1921:45Christian Johansen😊#2019-01-1921:55richiardiandreaThat blog is really comprehensive, I am thinking about making it a template now 😺 #2019-01-1922:00Christian Johansenhehe, please do. I considered making it a minimal github repo, but time didn't allow for it between shipping kids around, walking the dog and putting together IKEA furniture :shrug: 🙂#2019-01-2317:22rschmuklerHey all - does anyone know if anything became of the add-lib function described here http://insideclojure.org/2018/05/04/add-lib/ ?#2019-01-2317:23rschmuklerI noticed a branch (`add-lib`) on the github repo, but it's ~20 commits behind master. Is it still preferable to use it?#2019-01-2318:15alexmillerwe haven’t integrated it yet#2019-01-2318:16alexmillerI can merge up to master though if you want a master-compatible version of the branch#2019-01-2318:17alexmilleractually I did that recently, and add-lib branch is currently only like 1 commits behind master#2019-01-2318:22alexmillermerged, you can use sha e160f184f051f120014244679831a9bccb37c9de for latest#2019-01-2318:43rschmukler@alexmiller Thanks a ton! Heads up that I found a bug where a transient dependency wasn't reloaded into the classpath. I will see if it's fixed by the latest, and if not, open an issue.#2019-01-2318:43rschmuklerUnless you'd prefer me not opening issues for it, since it's not even part of a release yet#2019-01-2319:48alexmillerDid you mean transient or transitive?#2019-01-2319:49alexmilleradd-lib won’t do re-loading - it’s for additive use#2019-01-2321:36ghadi@seancorfield have you run into any issues with depstar and multi release jars?#2019-01-2321:38seancorfield@ghadi What do you mean by “multi release”?#2019-01-2321:39alexmilleras of java 9 you can make jars with different per-jdk sets of stuff in them#2019-01-2321:40alexmillerhttps://www.javaworld.com/article/3184029/java-language/java-9s-other-new-enhancements-part-4-multi-release-jar-files.html#2019-01-2321:40seancorfieldI haven’t tried to do anything with Java 9+ yet — we have a bunch of legacy code that won’t run on it.#2019-01-2321:42ghadiI hadn't considered MRJars and was curious if it panned out#2019-01-2321:44seancorfieldGiven that depstar doesn’t add a manifest — and filters out some manifest-related stuff as I recall — I don’t know how it would even go about that.#2019-01-2321:45ghadiI forgot whether it filtered anything or not from the Meta-Inf#2019-01-2321:45seancorfieldI started looking at adding basic manifest support but that seemed to lead me down the AOT path, at least for the Main-Class: entry when it wasn’t clojure.main … and I decided I didn’t have the stomach for that (yet).#2019-01-2321:45ghadiIf it works it's by accident#2019-01-2321:46seancorfield(defn excluded?
[filename]
(or (#{"project.clj"
"LICENSE"
"COPYRIGHT"} filename)
(re-matches #"(?i)META-INF/.*\.(?:MF|SF|RSA|DSA)" filename)
(re-matches #"(?i)META-INF/(?:INDEX\.LIST|DEPENDENCIES|NOTICE|LICENSE)(?:\.txt)?" filename)))
#2019-01-2321:47seancorfield(which seems pretty arbitrary — and I haven’t rationalized why it excludes all of that lot)#2019-01-2322:21dominicmDoes anyone have a firm reason that pack should implement a depstar like implementation? #2019-01-2322:21ghadiwhat does that mean?#2019-01-2322:26dominicmRather, is there a reason pack should adopt "jar Smashing"? #2019-01-2322:31hiredmanI've got a thing I worked for a little bit, but I don't think I have the time to go anywhere with it: https://gist.github.com/hiredman/d68cafb6aa8cea563c7b77d54f522421 instead of building uberjars (which can end up taking forever to transfer) it uploads individual jars/git repos/local deps only if they haven't already been uploaded and then deploys a file containing a built classpath pointing to those things#2019-01-2322:32hiredman(inspired by a talk rich gave somewhere about datomic ions)#2019-01-2407:22dominicmThat's really neat.#2019-01-2407:54plexushas anyone already come up with a thing that reloads deps.edn, adding any newly added libraries to the classpath?#2019-01-2408:19dominicmAlex's add-lib would probably be pliable to that. If you build it, I would be interested. #2019-01-2409:36thhellerFWIW I have done a few experiments with dynamically adding dependencies at runtime for shadow-cljs#2019-01-2409:37thhellerthe only case where this works reliably is when adding dependencies that don't depend on anything different than what is already loaded#2019-01-2409:37thhelleras soon as there are any version conflicts things get icky#2019-01-2409:38thhellereg. lib-a depends on X version 1.0.0 and is loaded. lib-b wants to be added but would cause X version 1.2.0 to be loaded instead of 1.0.0 if constructed normally#2019-01-2409:38thhellersince we can't easily unload version 1.0.0 you might now not be able to load properly#2019-01-2409:39thhellerunfortunately that scenario is quite common for bigger libs#2019-01-2409:39thhellerworks well enough for simple libs without deps though#2019-01-2409:39thhellergets even more complicated when accounting for npm deps 😉#2019-01-2409:39plexusI'm guessing pomegranate/alembic and friends also have this issue then#2019-01-2409:40thhellerdeps resolves version conflicts slightly differently so it is more prone to this#2019-01-2409:40thhellerbut yes same issue is present everywhere#2019-01-2409:40plexusI see#2019-01-2516:26alexmillerI don’t think deps is any “more prone” to this than pomegranate. it’s a fundamental issue with how libraries and classes are loaded on the jvm.#2019-01-2516:26alexmillerif you’re interested in clj on Windows, I have created #clj-on-windows as a working channel for anyone that wants to help with it#2019-01-2601:36matthiasHi @alexmiller. I uploaded a couple of patches for discussion on the tools.deps jira board. We are keen to get a few of those ironed out.
https://dev.clojure.org/jira/browse/TDEPS-52
https://dev.clojure.org/jira/browse/TDEPS-74
#2019-01-2601:38alexmillerthx, I prob won’t have time till next Friday#2019-01-3001:21lilactownis there a way to just get whatevers at master with git deps?#2019-01-3001:21lilactowntry to write a quick-try command for a lib#2019-01-3002:34alexmillerNo#2019-01-3002:35alexmillerBy design, we want to use stable versions for caching purposes#2019-01-3002:35alexmillerBut you can use the gitlibs lib to look up what sha master is#2019-01-3002:43alexmilleryou can even one-line it - clj comes with a built-in :deps alias that includes tools.deps (and tools.gitlib):#2019-01-3002:43alexmiller$ clj -A:deps -e "(require 'clojure.tools.gitlibs)" -e '(println (clojure.tools.gitlibs/resolve "" "master"))'
d86fec647b705b5d01d27c32322c9ccc9f65b77c
$ clj -Sdeps '{:deps {org.clojure/tools.gitlibs {:git/url "" :sha "d86fec647b705b5d01d27c32322c9ccc9f65b77c"}}}'
Checking out: at d86fec647b705b5d01d27c32322c9ccc9f65b77c
Clojure 1.10.0
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/conform (s/spec int?) 10)
10#2019-01-3002:48alexmillerrub some scripting on that and you’ve got something#2019-01-3003:45seancorfieldCould you combine that with the add-lib branch and dynamically load the master version into your REPL? (I can't remember whether add-lib works with git libs)#2019-01-3003:56alexmillersure#2019-01-3003:57alexmilleradd-lib works with any dep supported by tools.deps#2019-01-3004:07seancorfieldYeah, nice user=> (require '[clojure.tools.deps.alpha.repl])
nil
user=> (require '[clojure.tools.gitlibs])
nil
user=> (defn load-master [lib] (let [git (str "" lib ".git")] (clojure.tools.deps.alpha.repl/add-lib lib {:git/url git :sha (clojure.tools.gitlibs/resolve git "master")})))
#'user/load-master
user=> (load-master 'clojure/tools.trace)
Checking out: at fbba1a5012bae0ac4ed5cd704babf23e9e3fbc44
Downloading: org/clojure/clojure/1.4.0/clojure-1.4.0.pom from
Downloading: org/clojure/clojure/1.4.0/clojure-1.4.0.jar from
true
user=> #2019-01-3004:11seancorfield^ @lilactown#2019-01-3004:13lilactownClever! Thanks 😁#2019-01-3004:46seancorfieldI ❤️ Clojure 🙂#2019-01-3004:47seancorfield(interesting that clojure/tools.trace has a dependency on Clojure 1.4.0!)#2019-01-3004:56alexmillera lot of the contribs depend on the oldest version they can support#2019-01-3004:56alexmillerCI no longer tests anything older than 1.7 iirc#2019-01-3017:19jvtriguerosI haven't taken the plunge into using tools.dep yet, but absorbing via osmosis through Sean's code snippets! Thanks!#2019-01-3105:45seancorfieldI just updated my .clojure/deps.edn file on GitHub to add REBL support etc https://github.com/seancorfield/dot-clojure/commit/66f2b8fe1d1fd44ba42c79cd9489e7ac620e5fef#2019-01-3121:01borkdudeis it possible to access environment variables in deps.edn like $HOME? probably not..#2019-01-3121:01dominicmNope! Why do you ask? #2019-01-3121:02borkdudebecause I want to say ~/git/... in a local dep. just a convenience#2019-01-3121:04dottedmagIs it a transient thing for working on a local copy of a library? clj -Sdeps "...$HOME..."?#2019-01-3121:05borkdudeyup, can do it like that#2019-01-3121:55unbalancedhi all... so I've been attempting to transition over to the deps.edn -- first of all, is there a name for this style of build tools? aka lein vs boot vs ... what... deps?. In general it's great but there seems to be "something else going on". I'm seeing blog posts about a build.clj file but I don't see anything official on it. Can anyone explain what that's all about or have a good reference for it?#2019-01-3122:02lilactown@goomba leiningen and boot are fully-featured toolbelts for doing things like fetching and loading dependencies, building uber jars, running tests, etc.#2019-01-3122:03unbalancednod#2019-01-3122:04lilactowntools.deps is only about fetching and dependencies and building a classpath to run your JVM program. it doesn't handle any of the other things#2019-01-3122:04unbalancedokay, maybe I'm in the wrong place? is the clj tool not part of this and deps.edn?#2019-01-3122:05alexmillerit is#2019-01-3122:05alexmillerclj is about two things: making classpaths from transitive deps, and launching programs#2019-01-3122:05alexmillerdeps.edn describes the deps and ways to modify the making of classpaths#2019-01-3122:05alexmillerall other tasks are programs, so you launch a program to do the task#2019-01-3122:05unbalancedso far it's been great at those 😄#2019-01-3122:06alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/Tools#2019-01-3122:06unbalancedis that where these build.clj files I'm seeing come in?#2019-01-3122:06alexmiller^^ some tools#2019-01-3122:06alexmillerno one idea what build.clj is#2019-01-3122:06alexmillerbut it’s not deps.edn / clj#2019-01-3122:07lilactownbecause clj doesn't do all those other things, some people have decided to write their own script for e.g. building a JAR file, or bundling the CLJS code. instead of using a leiningen feature or plugin#2019-01-3122:07unbalancedhere's the only example I can find#2019-01-3122:07unbalancedhttps://github.com/clojure/clojurescript/blob/master/samples/string-requires-npm-deps/build.clj#2019-01-3122:07lilactownone might decide to call that script build.clj 🙂#2019-01-3122:08unbalancedokay so that stuff is essentially home-rolled scripting that some decide to call build.clj#2019-01-3122:08lilactownyeah. in this case, they wrote a script to bundle their CLJS code for deploying#2019-01-3122:09unbalancedinteresting. That is the one area in which I'm hurting right now -- I've been running lein in parallel with clj because I can't find a reliable way to do uberjars#2019-01-3122:09lilactownin leiningen, I believe there's a cljsbuild plugin to do that for you#2019-01-3122:09lilactownah yep. same for uberjars#2019-01-3122:09unbalancedmhm. Are uberjars not really a thing for clojure? I assumed that was the defacto standard for deployment#2019-01-3122:10alexmillerthey are very common. there are at least 3 tools in that tools page above that build them#2019-01-3122:12unbalancedindeed. Just trying to consolidate methodologies as much as possible.#2019-01-3122:13unbalancedthanks everyone for fielding questions 😄#2019-01-3122:13lilactownI think some people are moving away from them for Clojure, since there's not a ton of benefit you get other than improved startup time in most cases#2019-01-3122:14alexmillerthey are certainly convenient as a unit of deployment#2019-01-3122:16unbalancedwhat do they otherwise deploy with then? clj -m mything.core?#2019-01-3122:20lilactownah I was also conflating it with AOT. but currently I throw it all in a docker container because I already have infrastructure for that ¯\(ツ)/¯#2019-01-3122:21alexmillerAOT is orthogonal#2019-01-3122:59seancorfield@goomba FWIW, at work we use (my fork of) depstar to build all-source uberjars for pure convenience of shipping an "artifact" to a server; to run those we do java -cp path/to/the.jar clojure.main -m entry.point ... where entry.point is the namespace containing the -main we wish to run (our artifacts often contain several, related -main functions in different namespaces).#2019-01-3123:00seancorfieldAll of our dev/test process uses clojure (the core of clj) wrapped in a small bash script to simplify handling of command line options.#2019-01-3123:00unbalancedgotcha (worth a lot, as always!)#2019-01-3123:01seancorfieldWe do not AOT anything.#2019-01-3123:03seancorfieldIt's also worth mentioning, as background, that we started with lein back in 2011 (ish -- that was when we first took Clojure to production), and we switched completely to boot at the end of 2015 (I think, I'd have to go consult my blog to be sure), and then we switched completely to clj/`deps.edn` last year.#2019-02-0113:01mishais there good bare-bones template for clojure+clojurescript monorepo with tools.deps?#2019-02-0118:03dominicmEdge isn't quite bare bones, but is the only thing I know of without having looked much#2019-02-0200:21mishareading through the docs now, thanks#2019-02-0113:14misha{:deps {com.bhauman/figwheel-main {:mvn/version "0.2.0"}}
opieop#2019-02-0119:53eraadHi! I’m trying to use this library (originally built with Leiningen) from a clojure-cli project:
https://github.com/datil/xmldsig
I added a deps.edn but I have a couple of questions:
1. The lib has a src/main/java source folder. Do I need to add this as a “src” folder in the deps.edn?
2. I’m trying to import it using git deps, it adds to the classpath but I cannot seem to be able to import it.
Any recommendations on where to look?#2019-02-0119:54hiredmanyou won't be able to use it#2019-02-0119:54alexmillerlibraries with Java source can’t be used this way#2019-02-0119:54alexmillerclj is designed for libs with no “build” step#2019-02-0206:36matthiasThis is going to be game changing too - we're applying the idea in some really interesting ways.#2019-02-0206:42dominicmCan you share more about how you're able to leverage this to great effect? #2019-02-0119:55alexmillerwhich is most Clojure libraries, but not those with Java source#2019-02-0119:55alexmilleryou can use the built version as an artifact via Maven or via a locally built jar by pointing to it with a :local/root coordinate#2019-02-0120:07eraadCool @alexmiller @hiredman thanks!#2019-02-0121:49viestithis reminds me of https://dev.clojure.org/jira/browse/CLJ-2343#2019-02-0121:50viestiif gen-class would work at the repl, wouldn’t this mean a build-free support of libs using gen-class? 🙂#2019-02-0121:50viestidon’t know how many those would be though#2019-02-0121:54viestiOddly enough, I think I found a puzzle for Alex: https://github.com/metasoarous/oz/issues/44#issuecomment-459877612#2019-02-0121:56dominicm@viesti you have conflicting files in the class path, nothing can save you now. The ordering is pretty much random, oz is working by coincidence.#2019-02-0121:57dominicmFwiw, you could resolve this if you are a bit crazy by iterating over all of the matches.#2019-02-0121:57viestiyeah, the moment when you realise that the floor starts moving#2019-02-0121:58dominicmI'd recommend that public/index.html isn't used, and that a namespaced prefix is used instead.#2019-02-0121:59viestihum, that might be it#2019-02-0121:59viestisince might be difficult to fish out the conflicting dependency#2019-02-0122:01dominicmThe other library, json-to-html should also not use public/index.html#2019-02-0122:04viestiyeah, could point this out to the other lib#2019-02-0122:04viestimakes me think on using public/index.html in my own apps… 🙂#2019-02-0122:26dominicmYou should only use it there !#2019-02-0122:30viestiyeah, should report to the other dep actually, I think#2019-02-0122:34viestiseems that markdown-to-hiccup-0.6.1.jar might have unnecessary resources in it’s artifact#2019-02-0122:35dominicmIt's both really. Unless oz is an app.#2019-02-0122:35viestispend the day pulling a hairball of Ansible dependencies, now wondering how I ended up to another ball of strings in the evening#2019-02-0122:35viestithinking I’m good at catching balls of strings#2019-02-0206:43dominicmI've only learned this stuff from getting tangled too#2019-02-0206:44dominicmSome of us are destined for tangling#2019-02-0206:46dominicmI just discovered this also quite by accident: https://github.com/replikativ/hasch/blob/master/resources/public/index.html#2019-02-0206:46dominicmnow figwheel main doesn't work...#2019-02-0207:34viestiThe case of the creeping index.html... Trying to hold off thoughts of hijacking webapps by malicious classpath resource injection 🙀 #2019-02-0207:48dominicma lot of people unknowingly serve a /license.html as part of jetty iirc#2019-02-0207:48dominicmoh, no, it's an about.html#2019-02-0207:48dominicmI found out when I discovered the page on our site#2019-02-0210:40viestiheh, there really is such a resource, didn’t find it from served from couple of our apps#2019-02-0216:04dominicmDepends how you set it up, some setups serve html from the root :)#2019-02-0217:57viestiyup, duct template has nice default, which creates a folder with the app name for the resources#2019-02-0122:37viestiprobably both, although oz is higher in the hierarchy, but it can be a dependency itself#2019-02-0123:22viestialmost forgot, thanks for the advice! :) #2019-02-0411:33borkdudeIn a mono-repo is it possible to use a common deps.edn file with aliases that can be used from all sub repos?#2019-02-0411:33borkdude(without hacking $CLJ_CONFIG)#2019-02-0412:11dominicmNot that I've discovered.#2019-02-0412:11dominicmI don't believe it's possible.#2019-02-0412:24alexmillerNo, not currently#2019-02-0412:29borkdudeI’m experimenting with porting https://github.com/borkdude/boot-bundle to deps.edn, so in the future, we could migrate our boot stuff to deps.edn#2019-02-0412:30borkdudeThe idea is that you can define a bunch (bundle) of deps in a common file and then refer to a common name for that bundle in your deps.edn files’ :deps and/or :extra-deps.#2019-02-0413:24gphilipp@borkdude not sure if that covers your need, but @matthias.margush has created this tool: https://github.com/matthias-margush/aka#2019-02-0413:29borkdudethanks#2019-02-0417:53seancorfieldWe have a monorepo with dozens of subprojects -- and we use a small shell script to wrap clojure and use CLJ_CONFIG to point to the monorepo's deps.edn file (and then each subproject has its own deps.edn file). Works great. Any reason you don't want to do that @borkdude?#2019-02-0417:53seancorfieldBasically the shell script does cd path/to/subproject.clj; CLJ_CONFIG=../versions/deps.edn clojure ...args...#2019-02-0418:12richiardiandreadatapoint -> we have a monorepo and looking for a clean solution as well#2019-02-0418:12richiardiandreaor hints at how to contribute a solution#2019-02-0418:20borkdude@seancorfield the reason I thought I wanted to avoid that is that it messes with people’s own configs in their .clojure dir?#2019-02-0418:24alexmillersome possible answers, curious if these would help:
1) a clj flag to pass one or more config files that go “in the chain”#2019-02-0418:24alexmiller2) an env property to list or more additional config file locations#2019-02-0418:25alexmiller3) some kind of config file in .clojure/ (I’m guessing probably not ideal as this is for a group of projects but may not extend to everything you’re doing)#2019-02-0418:37dominicm1 makes this manual opt in everywhere, so users have to inform their tooling of this flag.
2 would require something like direnv, and would also require extensive editor integration.#2019-02-0418:38borkdudeanother way: :include "../parent.deps.edn"#2019-02-0418:38dominicm1 seems more tenable than 2 for something like cursive which has the most extensive analysis of deps.end files.#2019-02-0418:39borkdudeThat, together with a way of including the deps of aliases into another alias would be helpful to me#2019-02-0418:40ghosttoasterHey all. I'm having trouble getting a very simple deps.edn going. I keep on getting a message that opencv can't be found in mvn central, but I've added my custom repo where the jar can be found.#2019-02-0418:41eval20201 with less magic: clj --deps-file foo/deps.edn would only take foo/deps.edn into account. The current situation could be seen as clj --deps-file ~/.clojure/deps.edn --deps-file deps.edn ....#2019-02-0418:42seancorfield@borkdude when you're working on a monorepo project, I'd argue you want a more reproducible set of processes so I'd say you want .clojure to be omitted from deps.edn.#2019-02-0418:42dominicm@seancorfield I don't think that's any more true than a normal repo is it? #2019-02-0418:42borkdude@seancorfield people tend to put their cider-nrepl config into .clojure/deps.edn. I don’t want to disturb that#2019-02-0418:43borkdudeSince deps.edn is “just data”™ I’m inclined to rewrite a template to the deps.edn that will be used#2019-02-0418:43dominicmI also put Rebel, library overrides for environment specific bugs, benchmarking and more. All depending on the context I'm in.#2019-02-0418:44seancorfield@dominicm It's a matter of opinion 🙂 We've generally added stuff to our versions/deps.edn file that individual devs find useful. And we have a couple of "dev" namespaces that include REPL startup stuff. So there's really nothing in our .clojure/deps.edn files that would be useful in addition (we already have nrepl/socket REPL/REBL stuff in our project deps.edn files).#2019-02-0418:44dominicm@borkdude be aware that updating a deps.edn like that will erase comments. I wrote ednup to be a crappy solution to that, but it's not prime ready yet.#2019-02-0418:45borkdude@dominicm I’m exploring rewrite-clj using zippers, that will leave everything intact. https://github.com/Olical/depot does this too.#2019-02-0418:45seancorfieldThat said, I wouldn't object to an enhancement to tools.deps/`clj` that allows additional deps.edn files to be specified 🙂#2019-02-0418:45dominicm@borkdude that's what ednup does#2019-02-0418:46borkdude:extra-deps.edns#2019-02-0418:46borkdude@dominicm this one? https://github.com/SevereOverfl0w/ednup#2019-02-0418:46dominicm@seancorfield I work on a few monorepos (they're mono to their context), and updating my dev config in each of them would be a pain. I'd feel inclined not to commit my dev hacking and clutter git history, especially in projects where we have stringent reviews on code.#2019-02-0418:47dominicm@borkdude yup. You can see usage in my rewrite of resolve tags.#2019-02-0418:48borkdudeI’ve started experimenting with supporting deps.edn for boot-bundle here, also using rewrite-clj: https://github.com/borkdude/boot-bundle/blob/tools.deps/src/boot_bundle/rewrite.clj
but since it’s hard to settle down on an approach, I’m first looking if deps.edn itself could support this better#2019-02-0418:48dominicmIt's not a great library, but I plan to iterate on it intensely at some point, and it's great for simple things.#2019-02-0418:49dominicmOh, if you want examples, pack has an injector which uses rewrite directly.#2019-02-0418:51borkdudeI meant: figuring out the format/API is hard, not the actual programming 🙂#2019-02-0418:51borkdudeI want to support existing users of boot-bundle (including myself) and support a migration path from boot to deps.edn#2019-02-0418:52borkdudeanyways, being able to specify multiple extra deps.edns from the cmd-line is a good thing in my opinion#2019-02-0418:52borkdudeyou can always use -Srepro to exclude .clojure so that functionality is already there#2019-02-0419:02alexmillerit is still a pending todo to finalize what -Srepro means, esp in context of -Sdeps and other features. some of that is pending config file rework#2019-02-0418:53hiredmananother option, instead of having a super deps.edn for the monorepo, or generating dep.edns for subprojects might be something like a dependency linter, you give it some rules (deps on project x most version y) and run it over a directory and it checks all the deps.edn files it finds#2019-02-0418:54hiredmana linter could even be run against projects not in your monorepo to see how some other library might integrate#2019-02-0418:54borkdude@hiredman meh, then you would still have to update all those occurrences, or do you mean the linter would actually update the files?#2019-02-0418:54hiredmanno, you would have to update the files#2019-02-0418:55borkdudethe reason I wrote boot-bundle was to prevent having to do that, it became annoying#2019-02-0418:55borkdudeI would want one source of the truth and the include that in another project#2019-02-0418:56borkdudesome call it “managed dependencies” but I didn’t realize that when I wrote the lib#2019-02-0418:56hiredmanhow much dependency churn do you have?#2019-02-0418:57borkdudeso what I’m searching for is managed deps for deps.edn. Another way to accomplish this maybe is to have “empty” projects that declare their deps and then make a dependency on those with local/root.#2019-02-0418:58borkdudebut that gets out of hand too, I want it to be declarative and easy to modify#2019-02-0418:59borkdude@hiredman we have a couple of local libraries which all depend on subsets of the same dependencies. in some of those we use AOT. it’s nasty if you get the versions wrong, so we want them to be the same#2019-02-0418:59hiredmansure#2019-02-0419:03alexmillerwe always intended to support something like managed deps and I think that is a valuable use case#2019-02-0419:04alexmillerwe have not put as much thought into how that should look as with other stuff yet#2019-02-0419:04seancorfieldOur versions/deps.edn file is mostly :override-deps to "pin" dependencies for the whole monorepo. Each subproject declares its own dependencies as group/artifact {}. And we have dev/test/build aliases in versions/deps.edn.#2019-02-0419:05borkdude@alexmiller with being able to specify extra deps.edn configs from the cmd line + the ability to merge multiple aliases into one (:foo has :bar and :baz like with -R:bar:baz) I think we would be 99% there#2019-02-0419:05borkdudemaybe even 100%#2019-02-0419:05seancorfieldHaving four deps.edn in the chain instead of three would solve all our issues 🙂 (system, user .clojure, project root, subproject root).#2019-02-0419:07borkdudeproject/subproject seems arbitrary to me (different projects use different directory structures), being able to specify this yourself and number of times you can do this should be flexible#2019-02-0419:08hiredmanaot adds whole other wrinkle on top of things for monorepos, because you are depending on build output of a local project, not the local project (with clj doesn't directly support so much)#2019-02-0419:16hiredmando you aot as low down the deps tree as possible, or as high up as possible?#2019-02-0419:17borkdude@hiredman We wrote a lib that has some Java code which we compile up front. We don’t want to have the situation that we’re going to use different deps than what that was compiled against.#2019-02-0419:18hiredmanah interesting, built java not aot'ed clojure?#2019-02-0419:18seancorfield@borkdude Perhaps I'll rephrase that as "monorepo" and "project" level then instead of "project" and "subproject"? 🙂#2019-02-0419:19borkdudeWe also ran into trouble once with a service that was running clojure 1.8 which did different edn serialization than 1.9, that was also a problem. we just don’t want to think about this stuff too much and exclude any such possibility.#2019-02-0419:20borkdude@seancorfield boot-bundle supports loading bundles (managed deps) even from Clojars, it doesn’t matter how you organize this.#2019-02-0419:21borkdude@hiredman yes#2019-02-0419:28seancorfield@borkdude and each "bundle" would map to a separate deps.edn file? So you could have a folder for each "bundle" and then access them via multiple :local/root deps?#2019-02-0419:29borkdude@seancorfield how this is done currently in boot bundle is one keyword maps to one or more deps (or recursively, to another bundle). this is just EDN which is expanded dynamically in the scope of a build.boot file.#2019-02-0419:29seancorfieldFor us, the problem isn't really the deps themselves, but the aliases we want across the whole monorepo -- sort of like requiring every dev to have consistent .clojure/deps.edn files but just for the projects in the monorepo.#2019-02-0419:29borkdudewhere this bundle.edn file comes from is up to you, could be from clojars, a directory, etc.#2019-02-0419:30seancorfieldI haven't looked at anything in Boot for quite a while now. I hadn't heard of boot-bundle. Sounds useful in a deps.edn world.#2019-02-0419:32borkdudein boot this is easy to implement because you can actually program in the build file. in deps.edn I currently see no other way than to rewrite a deps.edn.template and expand it into a deps.edn. but with the options we mentioned in this conversation, I think it’s very close at hand#2019-02-0419:33borkduderelying on add-lib could also work, but I’m not too familiar with that and I think that’s alpha-alpha?#2019-02-0419:36alexmilleralpha-omega#2019-02-0419:42seancorfieldI recently added some aliases to my .clojure/deps.edn file that pull in that branch and the comments in my EDN file show how to write a load-master function that can resolve master from git repos and then add that library 🙂#2019-02-0419:42seancorfieldhttps://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L94-L107#2019-02-0423:57richiardiandrearead the whole conversation, our team is looking for the same answers @borkdude, maybe we can join forces 😄 Share the pain err your thoughts with me ok?#2019-02-0505:05borkdude@richiardiandrea Totally!#2019-02-0508:22Christian Johansen>Error building classpath. Manifest type not detected when finding deps for myapp/myapp in coordinate #:local{:root "/Users/christian/projects/myapp"}#2019-02-0508:23Christian JohansenI'm getting this on a project that used to work fine. The local dependency is a leiningen based project. Am I doing something wrong?#2019-02-0508:29Christian Johansenhmm, maybe I used to have a deps.edn file in that project? 🤔#2019-02-0512:57alexmillerProbably. You can force it with :deps/manifest :deps (and it’s tolerant of a missing deps.edn)#2019-02-0513:05Christian JohansenOk, good to know! Solved it by extracting project.clj deps into deps.edn. This is work in progress, and I've had a deps.edn file that wasn't commited to git that I probably git reset away 🙈#2019-02-0509:57borkdude@richiardiandrea Here’s an idea I have:
https://gist.github.com/borkdude/32d486805aabb1d8926270d4f12ff3dd
Feel free to shoot at it.
(cc @alexmiller @seancorfield @dominicm)#2019-02-0510:19dominicmIs there a jira ticket for managed deps?#2019-02-0510:21dominicmhttps://github.com/juxt/edge/issues/37 I was cleaning up issues, and realized this issue was related to what we were discussing last night 🙂#2019-02-0510:42borkdudeI’m willing to post my proposal into a JIRA ticket, but I thought it would be good to have a little screening first#2019-02-0510:44dominicmSure :) I actually wanted somewhere to direct the above issue to.#2019-02-0618:08timgilbertIs there anything like an offline mode for tools.deps? I'm finding my CI builds take 2-2.5 minutes per alias per build step just downloading maven-metadata.xml files from my S3 repo and clojars / maven-central, and I can't seem to cache the results#2019-02-0618:09timgilbertThis is separate from the actual .jar / .pom files which I'm able to cache with no problem (in ~/.m2)#2019-02-0618:13borkdudeWe use s3 wagon private in boot and I notice it downloads metadata more often than it should, which also slows down things there#2019-02-0618:14timgilbertAlso, these specific packages have become my arch-enemies as every time I run clojure I see this block of text:
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-kinesis/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-kinesis/maven-metadata.xml from
#2019-02-0618:16timgilbertI don't know if there's a TTL set to a ridiculously low value by AWS or what, but I just cannot avoid seeing that every single time if I've got those dpes#2019-02-0618:24alexmillerfile a ticket with as much info as possible#2019-02-0618:25aviI suspect the answer is no, but just to confirm: is there any way to have tools.deps compile Java code when consuming a Clojure library via git that also includes some Java code?#2019-02-0618:27alexmillerno#2019-02-0618:27aviAlrighty then, thanks!#2019-02-0623:03kennyIs there a way to see why a classpath refresh takes a long time with clj?#2019-02-0623:08alexmillerthere is a super secret verbose knob you could try#2019-02-0623:09alexmillerclj -Sdeps '{:aliases {:v {:verbose true}}}' -A:v#2019-02-0623:10alexmillerthe most likely culprit is slow I/O of course, which you can really spot only by observing where that pauses#2019-02-0623:10kennyThe weird part is it only occurs in one particular project.#2019-02-0623:11alexmillerthe next most likely is that tools.deps is doing more work than it should, which you may also be able to spot by seeing it redoing work#2019-02-0623:14kennyHard to tell what's expected and whats not. Expanding git deps takes a good 3-5s for this project.#2019-02-0623:16kennyThis project also has a fairly large dependency tree. The ~20s refresh gets a bit painful though.#2019-02-0623:19alexmillergit deps are stored in two parts - the git objects, and the working tree (per sha)#2019-02-0623:20kennyIt wouldn't be checking out anything new - everything is all on disk. Just changing the SHAs back and forth.#2019-02-0623:20alexmillerthe git objects are mostly append-only so a fetch should only need to be new stuff, small diffs#2019-02-0623:20alexmillerworking tree could be slower, but it’s all in the objects#2019-02-0623:21alexmillerI guess I would compare doing a git clone or fetch separately and see if it’s substantially different#2019-02-0623:22alexmillerthere’s caching in many places here - git objects, git working tree per sha, maven deps in maven repo, and classpath in cpcache#2019-02-0623:23alexmillerso I guess I’m curious what part of that isn’t cached when it’s slow (and why)#2019-02-0623:23kennyFor example, each one of these steps takes 3-5s:
Expanding compute/logger {:git/url
There are several of those.#2019-02-0623:24alexmilleris it correct that there is a newer sha?#2019-02-0623:25kennyYes, 319 is newer than 9f9f.
Just ran time on the -Spath command. It actually took 52s for a CP refresh.#2019-02-0623:27kennyI should clarify - that is one example of the messages that takes a while. AFAICT, there are no duplicates in the log.#2019-02-0623:55alexmillerso it should be fetching the git repo, then cloning the working tree. Are there maybe big binary objects in the working tree? One idea I had with gitlibs was to have the ability to narrow the working tree to just a subset of the full dir (so you could exclude like an images directory or something was irrelevant to the Clojure source)#2019-02-0623:56alexmillerstill not entirely sure that’s the slow bit from your description but maybe a testable hypothesis#2019-02-1117:24borkdudeRe managed deps:
https://dev.clojure.org/jira/browse/TDEPS-116#2019-02-1117:26borkdudeThat proposal introduces the ability to have managed deps with minimal change to the deps.edn spec (don’t know about the impl)#2019-02-1209:34martinklepschWhen using :local/root with a .jar will that resolve transitive dependencies of the jar by inspecting the pom.xml inside that jar?
And is it standard that pom.xml is included in the jar?#2019-02-1209:37borkdudeIs that supposed to work? I’m surprised#2019-02-1209:38borkdudeYeah, it’s documented: Local jar: {:local/root "/path/to/file.jar"}. TIL#2019-02-1209:39martinklepschYeah, it's pretty neat. I just wasn't aware that pom.xml is always(?) part of the jar so thought that this couldn't work with transitive deps.#2019-02-1213:46alexmillerIt is normally part of the jar.#2019-02-1214:22stijnwhat might be the reason that it takes a very long time to get a REPL up? I just upgraded my ArchLinux and before it took a few seconds, now, minutes. I believe the only change is the upgrade of the jdk 8.u192-1 -> 8.u202-1.#2019-02-1214:24stijnIf I don't include the 'dev' profile, which has some machinery with tools.namespace (and a logback.xml) in it, it goes back to seconds.#2019-02-1214:33alexmillerwell the main thing it’s doing is network I/O so that would be my default area of concern#2019-02-1214:34alexmillerdo some stack traces with ctrl-\ while it’s working and see what it’s doing#2019-02-1215:07alexmiller8.u201 (and 202) include new buffer overrun checks in Linux, not sure if that would affect visible perf: “Additional safeguards to protect against buffer overruns in native code have been enabled on Linux. If a buffer overrun is encountered the system will write the message “stack smashing detected” and the program will exit. Issues of this type should be reported to your vendor.”#2019-02-1216:28stijnif I switch to java 11 it's back to 'seconds'#2019-02-1216:30stijnit looks like the compilation of namespaces that are required from user is slow. when these are uncommented, it's fast on 8.u202 too#2019-02-1217:20alexmillerhmm, still not clear to me what the difference is#2019-02-1217:21bronsacould you post your user.clj maybe?#2019-02-1307:26stathissiderisdoes the Clojure runtime have access to the aliases that were passed? Can my code access this information somehow?#2019-02-1307:31dominicmThere are some system properties set. I think only clojure.libfile#2019-02-1307:31dominicmWhy do you need to know? #2019-02-1307:50stathissiderisI’d like to know whether its a figwheel run or not#2019-02-1307:59dominicm@stathissideris everyone is going to name their Figwheel alias differently. I'd check if the namespace is loaded/can be required, and depending on context, check if the state var is bound.#2019-02-1307:59dominicmYour code might work with some lein setups too that way#2019-02-1308:04stathissiderisgood point about about naming, but I need this for my specific setup, I’m not making a generic tool, so hardcoding it would be ok#2019-02-1308:06stathissiderisIs is possible to combine -e and -m to inject a var maybe?#2019-02-1308:13dominicmI do this in edge using :jvm-opts#2019-02-1308:14dominicmJvm opts compose, main opts do not.#2019-02-1308:15dominicmUsing -D in jvm opts that is.#2019-02-1308:17stathissiderisoh thanks, I’ll check it out#2019-02-1313:40dominicmI think there's a NPE in t.d.a:
Error building classpath. nil
java.lang.NullPointerException
at clojure.tools.deps.alpha.extensions.maven$eval638$fn__641.invoke(maven.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$canonicalize_deps$fn__1027.invoke(alpha.clj:74)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
at clojure.core$reduce.invokeStatic(core.clj:6827)
at clojure.core$reduce.invoke(core.clj:6810)
at clojure.tools.deps.alpha$canonicalize_deps.invokeStatic(alpha.clj:73)
at clojure.tools.deps.alpha$canonicalize_deps.invoke(alpha.clj:71)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:170)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:236)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:218)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:491)
at clojure.main$main_opt.invoke(main.clj:487)
at clojure.main$main.invokeStatic(main.clj:598)
at clojure.main$main.doInvoke(main.clj:561)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:37)
this may be time sensitive as I'm using a just-deployed coordinate.#2019-02-1313:40dominicmorg.webjars.bower/react {:mvn/version "15.4.2"}
org.webjars.bower/es6-promise {:mvn/version "4.0.5"}
org.webjars.bower/fetch {:mvn/version "0.9.0"}
org.webjars.bower/graphiql {:mvn/version "0.11.11"}
org.webjars.npm/graphiql-subscriptions-fetcher {:mvn/version "0.0.2"}
org.webjars.npm/subscriptions-transport-ws {:mvn/version "0.8.3"}
these are the deps I just added#2019-02-1313:42dominicm org.webjars.npm/graphiql-subscriptions-fetcher {:mvn/version "0.0.2"} this appears to be the exact problem dep#2019-02-1313:51alexmillerruntime does not have access to the aliases that were used @stathissideris#2019-02-1313:53stathissideris@alexmiller thanks, I ended up using -D to indicate the alias via properties as @dominicm suggested #2019-02-1313:53alexmillerI can repro the npe, will take a look. shouldn’t be timing-related#2019-02-1313:54alexmillerthere’s a version range in the transitive deps, wouldn’t surprise me if that’s involved#2019-02-1313:55dominicmthat seems more likely, yeah. I was worried that we'd caught maven in an inbetween state and it was causing an NPE.#2019-02-1313:56alexmillerI think it’s finding no matches in the version range#2019-02-1313:56alexmillerso code could definitely be tightened up around that to not npe but it will just not find the dep then#2019-02-1313:57dominicmyeah. makes sense. webjars is a bit weird like this I guess 🙂#2019-02-1313:57dominicmI can add explicit graphiql dep here.#2019-02-1313:58alexmillerorg.webjars.npm/graphiql-subscriptions-fetcher {:mvn/version "0.0.2"} depends on org.webjars.npm/graphql {:mvn/version "[0.9.1,0.10)" but I don’t see anything older than 0.10.1 in maven central#2019-02-1313:58alexmillerhttps://repo1.maven.org/maven2/org/webjars/npm/graphiql/#2019-02-1313:58alexmillerI’ll fix the npe#2019-02-1313:59dominicmwebjars is a bit weird. You basically are deploying npm things. So if nobody has deployed graphiql 0.10, then it won't be there.#2019-02-1313:59dominicmadding an explicit graphiql dep hasn't helped though, annoyingly. So I guess I will deploy 0.10 via http://webjars.org#2019-02-1313:59alexmillerdoes npm actually have jars in it?#2019-02-1314:00dominicmwebjars takes stuff from npm, and puts it in a jar#2019-02-1314:00alexmillertdeps could have an extension to use webjars directly#2019-02-1314:00dominicmdo you mean npm directly? I don't know how you'd use webjars directly 🙂#2019-02-1314:01dominicmbut either way, that would be a huge +1 from me#2019-02-1314:02alexmillerwell, you would need to tell me more things for me to understand what’s possible, but anything that can be a source of named, versioned artifacts is a potential target#2019-02-1314:08dominicmgood question, admittedly I don't fully understand how webjars works.#2019-02-1314:08alexmillerwell, webjars seems to have a a cdn? whatever step you’re doing seems like some sort of mindless wrapping task#2019-02-1314:08alexmillerprograms are good at those#2019-02-1314:09dominicm> All of the WebJar contents are available on the public jsDelivr CDN. Just prefix //cdn.jsdelivr.net/webjars/{groupId} in front of your static asset URLs. For instance, if using the org.webjars : jquery WebJar and your local URL to jquery.js is /webjars/jquery/2.1.0/jquery.js then the CDN URL would be: //cdn.jsdelivr.net/webjars/org.webjars/jquery/2.1.0/jquery.js
So I guess the contents of the jars are available via the CDN, rather than the jars themselves.#2019-02-1314:09dominicmthe actual jars go to maven.#2019-02-1314:09dominicmthen you can request a new jar to go to maven via clicking buttons on http://webjars.org - that triggers an automated process#2019-02-1314:13alexmiller“clicking buttons” :)#2019-02-1314:14dominicmis your proposal that instead of deploying to maven, a webjars-like thing would build the jars and stick them in ~/.local/cache/webjars/foo.bar.1.jar ?#2019-02-1314:15alexmillerwhy do you even need jars? they’re just files on the classpath right?#2019-02-1314:16alexmillerit would be helpful to do something that actually helped people down the line as they build and deploy stuff#2019-02-1314:17alexmillerI am too ignorant about cljs / npm / node / etc to not know what would be useful#2019-02-1314:17dominicmyou're right. jars not needed.#2019-02-1314:17alexmillernpm is a central repository. it has stuff you want in your app. tools.deps has an extension point to plug in code to get stuff and put it on your classpath (and determine its dependencies)#2019-02-1314:17dominicmfwiw, this is pretty unrelated to cljs.#2019-02-1314:18alexmillersee how ignorant I am? :)#2019-02-1314:18dominicmI think npm is a good target for t.d.a. So is bower.#2019-02-1314:18dominicmmaybe not bower, it's kinda old now.#2019-02-1314:19alexmilleryeah, anything that didn’t come out this month in js is old :)#2019-02-1314:20alexmillerI guess coming back to a problem statement - this is like js stuff that you want to serve as static resources from your app?#2019-02-1314:24alexmillerdoes npm really not have any kind of namespacing?#2019-02-1314:27alexmilleroh, it has scopes, I see#2019-02-1314:28alexmillerwell, seems like the info necessary exists if you want to file a ticket defining what’s useful#2019-02-1314:29dominicm> I guess coming back to a problem statement - this is like js stuff that you want to serve as static resources from your app?
Exactly!#2019-02-1314:30dominicmAlthough, admittedly, I'd personally be quite happy with cdn integrations.#2019-02-1314:30dominicmbower predates npm afaik. It was a big step forward at the time, but it's not really used. npm got more traction and was dominant by the time a majority of companies turned their attention to js.#2019-02-1314:31dominicmbower doesn't recommend you use bower so ¯\(ツ)/¯#2019-02-1314:31dominicmI'll write down that I want to create this, and have a note for myself to think a little more about what I want 🙂#2019-02-1314:32dominicm@alexmiller while I have you, is there a workaround for this graphiql issue? bintray appears to be having issues, and I can't get an older graphiql to deploy. :exclusions hasn't worked, nor has an explicit graphiql dependency on 0.11.11.#2019-02-1314:35alexmillernot sure I have a good option for you#2019-02-1314:36alexmillerif you made webjars deploy version 0.9.1 to maven central that would let it proceed#2019-02-1314:37dominicmUnfortunately, that's failing 🙂 I thought you might say that though. It's okay#2019-02-1314:40alexmillerkind of an interesting failure mode. would be better if it could defer long enough to be overridden#2019-02-1314:40alexmillerin this particular case, the only reason it’s resolving early is due to the version range I think, trying to narrow that down to single version#2019-02-1316:43dominicm#2019-02-1316:43dominicmgraphiql vs graphql#2019-02-1317:17dominicmYou hit an NPE whenever you have a transitive dependency which doesn't exist. I'm seeing it with a version pinned to [1.0.3] (maybe that is a range though?)#2019-02-1317:19dominicmActually, no. I'm confused.#2019-02-1317:19dominicmhttps://search.maven.org/artifact/org.webjars.npm/graphql/0.9.1/jar depends on https://search.maven.org/artifact/org.webjars.npm/iterall/1.0.3/jar but I'm still hitting the exact same NPE.#2019-02-1317:20dominicmI've rm -rf .cpcache, and used -Sforce, just in case.#2019-02-1317:20dominicm❯ clj -Sdeps '{:deps {org.webjars.npm/graphql {:mvn/version "0.9.1"}} :aliases {:v {:verbose true}}}' -A:v -Sforce
Initial deps to expand:
{org.clojure/clojure #:mvn{:version "1.10.0"},
org.webjars.npm/graphql #:mvn{:version "0.9.1"}}
Expanding org.clojure/clojure #:mvn{:version 1.10.0}
=> include, pin top dep
Expanding org.webjars.npm/graphql #:mvn{:version 0.9.1}
Error building classpath. nil
java.lang.NullPointerException
at clojure.tools.deps.alpha.extensions.maven$eval638$fn__641.invoke(maven.clj:41)
#2019-02-1317:35dominicm[ERROR] Failed to execute goal on project my-app: Could not resolve dependencies for project : Failed to collect dependencies at org.webjars.npm:graphql:jar:0.9.1 -> org.webjars.npm:iterall:jar:[1.0.3]: No versions available for org.webjars.npm:iterall:jar:[1.0.3] within specified range -> [Help 1] maven is struggling too#2019-02-1317:36dominicmOh. Interesting. It is fine with it as a top-level dependency, but not as a transitive dep.#2019-02-1318:17alexmiller[1.0.3] is a version range. from Maven’s perspective just 1.0.3 by itself is a specific version#2019-02-1318:20alexmillerI don’t think there’s any difference with top vs transitive - you probably just have it in your local Maven repo at ~/.m2/repository now#2019-02-1318:22alexmillerit works for me as a transitive dep, and as a top-level dep with specific version, and as a top-level dep with version range [1.0.3]#2019-02-1318:22alexmiller(clearing all caches before each one)#2019-02-1318:24alexmiller$ rm -rf ~/.m2/repository/org/webjars
$ clj -Sdeps '{:deps {org.webjars.npm/graphql {:mvn/version "0.9.1"}}}' -Sforce
Downloading: org/webjars/npm/graphql/0.9.1/graphql-0.9.1.pom from
Downloading: org/webjars/npm/iterall/maven-metadata.xml from
Downloading: org/webjars/npm/iterall/1.0.3/iterall-1.0.3.pom from
Downloading: org/webjars/npm/graphql/0.9.1/graphql-0.9.1.jar from
Downloading: org/webjars/npm/iterall/1.0.3/iterall-1.0.3.jar from
Clojure 1.10.0
user=>#2019-02-1318:27alexmillerwhen you deploy to Maven central it can take 10 minutes to an hour to show up for download, so maybe it just wasn’t there yet#2019-02-1318:37dominicmI'm a bit suspicious that perhaps version ranges are searched somewhere different than specific versions.#2019-02-1318:41alexmillerthey aren’t#2019-02-1318:41alexmillerbut there is an extra piece of code to explicitly resolve them in tdeps#2019-02-1318:42alexmillerif you haven’t, you should try selectively deleting part of your ~/.m2/repository/org/webjars/…#2019-02-1318:42alexmillerthere are metadata files about versions that are cached there and only updated once every 24 hrs#2019-02-1318:42alexmiller(this is a setting we don’t currently surface - if you’re using mvn, you can pass -U to force an update)#2019-02-1318:43dominicmIs it possible that metadata would store [1.0.3] separately from 1.0.3?#2019-02-1319:01alexmillerit just stores known versions and then uses that to resolve#2019-02-1319:01alexmillerbut if you had already filled that cache for the day and the repository said that version didn’t exist, it’s going to keep telling you that#2019-02-1319:01alexmillerit will not go re-check the repository#2019-02-1319:02alexmillerthis file in particular: ~/.m2/repository/org/webjars/npm/iterall/maven-metadata-central.xml#2019-02-1319:03alexmillerlast check timestamps are in ~/.m2/repository/org/webjars/npm/iterall/resolver-status.properties#2019-02-1319:59stijn@alexmiller on the slow REPL startup of the latest java 8 (u202) on Linux, looking at the stack traces, it spends a lot of time compiling some java-time things (https://github.com/dm3/clojure.java-time). One CPU goes to 100% during this and after that stack trace is gone, the REPL starts.#2019-02-1320:08hiredmanjava-time does lot of meta programming stuff, finding types, doing a graph traversal to find paths between types#2019-02-1320:08hiredmanso I could imagine types changing between jdk versions#2019-02-1320:13dominicmblugh, after all this work, one of my transitive deps doesn't work because it has a custom gql license which webjars won't automatically repackage because it doesn't recognize. If we were a direct npm client this would be avoided. Very tempting proposition really.#2019-02-1320:13dominicm@alexmiller if I wrote an npm client with an EPL license in clojure, would it be candidate for use in t.d.a? Or would you prefer it was under the clojure/core umbrella?#2019-02-1320:42alexmillercould be an external lib (we use others already). I think there are some out there already. but maybe the amount of access needed by tda is so small to not even be worth that, not sure.#2019-02-1320:43alexmillerI guess you probably want a cache ala tools.gitlibs though and that’s something we’d want outside tda#2019-02-1320:46dominicmI haven't really looked at the npm client details yet. Might be worth seeing if we can share the yarn cache. Plus I'm a bit uncertain about how to handle http. #2019-02-1320:47alexmillerwell I think you have the same options as gitlibs - shell out to npm or make a direct implementation#2019-02-1320:49alexmillerpros and cons. might be worth gathering some more information about the things you need to implement:
1) given a name/version, determine that lib’s deps (don’t need transitive, just shallow)
2) given a name/version, download the bits somewhere#2019-02-1320:49dominicmNpm is far more amenable I think. Yarn is a reimplementation of the npm client. Plus, npm doesn't have a global cache, and because of the hierarchical nature, you probably want more control.#2019-02-1320:49dominicmI suppose you have to figure out auth, which is the recurring issue.#2019-02-1320:53alexmillerwell public stuff is fine I assume#2019-02-1320:53alexmillerfor creds, we already have a means to do this via the maven settings.xml on a per-repo basis#2019-02-1320:53alexmilleradmittedly kind of weird to put npm auth there#2019-02-1320:19stijn@hiredman, the weird thing is that it's only a minor version upgrade (from 8.u192 to 8.u202). With 192 the repl starts in seconds, with 202 it takes minutes#2019-02-1320:21ghadiI know it's aside from your problem @stijn but I wouldn't use that library, to be perfectly honest.#2019-02-1320:21ghadijava.time is perfectly approachable#2019-02-1320:21ghadijava.time is perfectly approachable#2019-02-1320:22dominicmhttps://juxt.pro/tick/docs/index.html makes java.time quite nice, and it works in cljs 🙄 (I work for JUXT)#2019-02-1320:29seancorfieldThat's a matter of opinion @U050ECB92 🙂 I find clojure.java-time really does produce cleaner-looking code in a lot of situations.#2019-02-1321:17ghadiclojure.java-time seems to have a steep cost in classloading and metaprogramming#2019-02-1321:19seancorfieldIt's a one-off cost, the first time you use certain constructors. It's fast after that.#2019-02-1321:20seancorfieldThe readme even shows how you can "warm up" that stuff.#2019-02-1321:20seancorfieldI was a bit surprised that it caused such a "blip" on 202(?) since it's fine on 192 (and on subsequent builds) according to the OP.#2019-02-1408:47stijnI will investigate some more when I have the time. For now, went back to 192.#2019-02-1320:21ghadithe actual JDK package java.time#2019-02-1320:24hiredmanmaybe check to see if you have org.threeten.extra.Temporals and or org.joda.time.DateTime on your classpath, and if calling (Class/forName ...) with those class names changes behavior on the different jdk versions#2019-02-1408:50stijnyes org.joda.time.DateTime is on the classpath. will check later what the impact is#2019-02-1320:46alexmillertools.deps.alpha 0.6.488 and clj 1.10.0.414 now available - https://groups.google.com/forum/#!topic/clojure/5EWvlgC4C9U#2019-02-1320:46alexmillerjust a couple small enhancements, no biggie#2019-02-1320:46dominicmWoah, very quick, thanks! #2019-02-1320:49alexmillerhadn’t released in a while#2019-02-1321:25seancorfieldThat Groups post hasn't gone out via email again, as far as I can see @alexmiller#2019-02-1321:27seancorfield(I just replied to it on the web which should cause it to be seen via email 🙂 )#2019-02-1322:12alexmillerI sent that one from a different account#2019-02-1419:40avilooks like the brew PR has been merged — I just ran brew upgrade and my Clojure install got upgraded to 1.10.0.414#2019-02-1419:52alexmilleryep, it's out there now#2019-02-1420:00avi🎉 thanks Alex! aw_yeah#2019-02-1420:08alexmillerjust noticed there are some homebrew download stats at https://formulae.brew.sh/formula/clojure#2019-02-1420:51aviThat’s nice! Looks like you can get the stats as data in JSON format too: https://formulae.brew.sh/api/formula/clojure.json#2019-02-1502:56mfikesRegarding the slowdown in REPL startup times when going from 1.8.0_192 to 1.8.0_201, I encountered this on macOS. Details here (with essentially a trivial repro) https://clojurians-log.clojureverse.org/cljsrn/2019-02-05#2019-02-1716:34ChrisHave you seen this?
http://insideclojure.org/2019/02/16/journal/#2019-02-2014:53zetafishDoes anyone know how to change the local maven repo folder when running clj so that jars are downloaded and searched for in a different folder than ~/.m2#2019-02-2015:08alexmillerin deps.edn, set :mvn/local-repo "/foo/bar" at top level#2019-02-2016:23zetafishThanks @alexmiller that works. Sadly enough it seems not possible to put this option in the :aliases. My problem is that I want the maven repo to be local to my source code when I run unit tests on my CI server. While when developing locally I want to just use the default ~/.m2. Are there plans to allow the key also in the :aliases? Or maybe I should revert to tricks like rewriting the deps.edn file on the CI server prior to running tests etc (ugh)#2019-02-2016:25alexmilleryou can use -Sdeps '{:mvn/local-repo "whatever"}' just on CI#2019-02-2016:25zetafishCool! I am just a beginner with tools-deps 🙂#2019-02-2016:59dominicmMy friend got this error when using brew on his mac, I don't use mac, so don't really understand why gcc is related to clojure at all?
Error: An exception occurred within a child process:
CompilerSelectionError: clojure cannot be built with any available compilers.
Install GNU's GCC
#2019-02-2017:00alexmillerseems bad#2019-02-2017:00alexmillerbrew doctor is the generic "why is brew broken?"#2019-02-2017:00alexmillerbrew install clojure should not require anything to be compiled#2019-02-2017:01alexmillerbrew update updates brew itself#2019-02-2017:02dominicmDoes brew generally work without xtools?#2019-02-2017:07dominicminstalling xtools solved it, so I guess that's a dependency? /shrug#2019-02-2017:11alexmillerI never understood how xtools is exactly involved, I guess to allow brew to compile stuff#2019-02-2017:13dominicmyeah, but, unless it compiles rlwrap from source, I have no idea what it would be building.#2019-02-2017:28alexmillerMe neither#2019-02-2116:39stathissiderisdoes this look wrong somehow? It’s an alias in my deps.edn
{:main-opts ["-e" "(require 'dev) (dev/seed-db)"]}
and it throws a
Exception in thread "main" clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: EOF while reading, starting at line 1#2019-02-2116:59alexmilleryeah, the inner string is not going to end up getting quoted right#2019-02-2117:02stathissiderisso it just needs more quoting? {:main-opts ["-e" "\"(require 'dev) (dev/seed-db)\""]}#2019-02-2117:03stathissideristhat doesn’t work either#2019-02-2117:03alexmillerno, that probably won't work either due to how this gets propagated. this is a known issue#2019-02-2117:03alexmilleryou can try the "corfield comma" (comma instead of spaces)#2019-02-2117:04alexmiller{:main-opts ["-e" "(require,'dev)(dev/seed-db)"]}#2019-02-2117:04stathissiderisyes, that does work, thank you very much — I don’t think I would have found it myself!#2019-02-2117:39seancorfieldsomeone summoned me 🙂 @stathissideris It's usually around this time I point people at my dot-clojure file for examples of :main-opts https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#2019-02-2117:42stathissiderisI’ve been slowly figuring out many of these myself, so many thanks for this treasure trove!#2019-02-2208:13ChrisThis is great! For your rebl-11 alias, does that remove the need to install OpenJFX? #2019-02-2209:17ChrisWell, it does! I just tried 🙂#2019-02-2402:55maxpHow to make uberjar like depstar does but with aot and manifest, optionally excluding sources?#2019-02-2413:09ChrisCambada might be able to do what you need. It can do no sources and AOT, not sure about manifests.
https://github.com/luchiniatwork/cambada/blob/master/README.md#2019-02-2416:45borkdudeYou might also get away with this approach, using leiningen as a lib?
https://github.com/oakes/full-stack-clj-example#2019-02-2500:56albert.laihttps://github.com/EwenG/badigeon#2019-02-2509:05ChrisWow Badigeon looks great, nice. #2019-02-2514:01maxpCambada looks great but there was some critical issue - it failed to make jar with cheshire 5.8.1 that comes with new Jackson, with cheshire 5.8.0 it wokrs well.#2019-02-2514:05maxpOk, I'll try badigeon later.
At that time I just do (compile ') and depstar classes into jar 🙂#2019-02-2521:04aviCambada has been working well for my project.
Here’s my deps.edn alias: https://github.com/FundingCircle/fc4-framework/blob/master/tool/deps.edn#L68#2019-02-2609:19maxpIt works well when I use cheshire 5.8.0 in project dependencies but fails with cheshire 5.8.1!
The difference is in version of Jackson used by Cheshire.#2019-02-2512:08souenzzofew days ago i saw it on twitter a project that use's tools-deps + lein as library to generate uberjar but I lost the link 😞#2019-02-2512:08dominicmNot lein-tools-deps do you mean?#2019-02-2512:08dominicmor pack.alpha / depstar maybe?#2019-02-2512:11borkdudeI just mentioned it in the thread above: https://github.com/oakes/full-stack-clj-example#2019-02-2720:20pbrownHey there! I have a small question about intended use/behavior:
Should libraries declare [:paths ["src"]] in their deps.edn when they use the default?
If my deps.edn includes a top-level [:paths ["bogus/sources"]] entry, and I bring a lib in via :git/url that does not declare :paths in its own deps.edn, its :paths-derived classpath entry becomes ~/.gitlibs/libs/$GROUP_ID/$ARTIFACT_ID/$SHA/bogus/sources, which doesn't exist.
For example:
clj -Srepro -Sdeps '{:paths ["bogus/sources"] :deps {com.bhauman/figwheel-core {:git/url "" :sha "840f05547935f65a28c5a3c640a9473961da5636"}}}' -Spath | tr : '\n' | grep figwheel
# => /home/phil/.gitlibs/libs/com.bhauman/figwheel-core/840f05547935f65a28c5a3c640a9473961da5636/bogus/sources
#2019-02-2720:20pbrownOr is this something tools.deps could/should do (would it be safe to assume we all use the same /usr/local/lib/clojure/deps.edn)?#2019-02-2720:24alexmillerThis a known bug#2019-02-2720:24alexmillerWhere the current project paths are used with dependency projects on the file system#2019-02-2720:28alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-52#2019-02-2720:28pbrownah ty, missed it the first time through. It has the same repro!#2019-03-0202:47nbenketafHi there, can I mix clojure and java files in the same tools deps project ?
Like :path ["src" "java-src"] where src contains clojure source files and java-src java source files, and then import the classes from java source files directly in a clj source file ?#2019-03-0202:51hiredmanno#2019-03-0202:59nbenketafok thanks, are there some external tools/libs that can achieve that or is it planned in the tools deps roadmap ?#2019-03-0203:01hiredmanas far as I know it is positively not on the road map, java requires compilation before you can use it, a build step, and tools.deps is positioned as "not a build tool"#2019-03-0203:02hiredmanI would be kind shocked if there isn't a tool someone wrote for it, but I don't know of one off hand#2019-03-0203:08nbenketafMaybe I can handle that with the “full-stack-clj-example” above, integrating leiningen as a lib, thanks anyway 🙂#2019-03-0203:52seancorfield@nbenketaf Or you could, you know, run javac on your Java code, and then run clj...#2019-03-0204:01seancorfieldAnother option is this: {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "126177737c715256574159465d423f4347204758235c"}, :content ("[email protected]")}#2019-03-0205:00nbenketaf@seancorfield thank you for the inputs, here is the larger problem I am trying to solve : I have a forked project A initially managed with lein which includes some java files. I tried to convert this project to a tools deps one, basically I added a deps.edn file. Then I have another project B that declares a deps to project A with a git/url coordinate, and it’s in project B where I need the compiled java code. I guess I should package my forked project A, and use the mvn coordinate instead#2019-03-0205:09seancorfieldYeah, if a dependent project has code that needs to be compiled, either you need to package it as an "artifact", or you'll need to bring it down as source and compile stuff yourself.#2019-03-0205:10seancorfieldJava source projects pretty much require compiled artifacts for use. Clojure is trying to get away from that by allowing source projects to be used as-is.#2019-03-0210:18nbenketafOk i see, thank you for taking the time answering 🙂#2019-03-0421:00rschmuklerIs anyone using https://github.com/clojure/tools.deps.alpha/tree/add-lib effectively? I'm getting a NullPointerException on clojure.tools.deps.alpha.util.maven/make-session (maven.clj:141) - seemingly since around the time the zero-day vulnerabilities in the JVM got fixed (but I might be conflating things). No worries if the answer is "this branch isn't supported, don't use it"#2019-03-0421:01alexmillercould you be more specific about what you're doing?#2019-03-0421:01rschmuklerCalling add-lib causes that to be thrown#2019-03-0421:02rschmuklerHmm actually though, not all the time#2019-03-0421:02alexmillerso, that's a branch, what commit are you using, what is the full call, etc. give me a repro.#2019-03-0421:03rschmuklercalling it with clojure.java-time {:mvn/version 0.3.2} seems to be the issue.#2019-03-0421:03rschmukler@alexmiller will do - give me a moment to try and create a minimal repro repo#2019-03-0421:04alexmilleryou don't need a repo - you can put it all in an clj -Sdeps call to start, then whatever expression you're doing#2019-03-0421:16alexmiller$ clj -Sdeps "{:deps
> {org.clojure/tools.deps.alpha
> {:git/url \""
> :sha \"e160f184f051f120014244679831a9bccb37c9de\"}}}"
Checking out: at e160f184f051f120014244679831a9bccb37c9de
Clojure 1.10.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'clojure.java-time {:mvn/version "0.3.2"})
true
user=> (require 'java-time)
Syntax error (IllegalAccessError) compiling at (temporal.clj:324:1).
class clojure.lang.PersistentUnrolledVector$Card1 tried to access field clojure.lang.APersistentVector._hash (clojure.lang.PersistentUnrolledVector$Card1 is in unnamed module of loader clojure.lang.DynamicClassLoader @7a0e1b5e; clojure.lang.APersistentVector is in unnamed module of loader 'app')#2019-03-0421:17rschmukler@alexmiller I'm not even getting that far, although I can't seem to reproduce the issue in a new repo so now I'm trying to recreate more and more of the state in my user.clj as to what's happening before I invoke add-lib.#2019-03-0421:18alexmillerI get that defaulting to Clojure 1.10. Not sure what clojure.java-time or the dependency clj-tuple assume with clj or java version#2019-03-0421:19rschmuklerInteresting. I've got clojure.java-time working w/ a 1.10 repo, although I added it via deps.edn, not dynamically via add-lib - so I haven't tested it.#2019-03-0421:21alexmillerclj-tuple seems to be compiled Java classes, certainly possible that doesn't play nice with add-lib#2019-03-0421:21alexmillerthat error doesn't match what you described though re NPE#2019-03-0421:21rschmuklerI'm seeing it w/ other libs as well - but I'm hitting the error while invoking add-lib, not after (and trying to require it)#2019-03-0421:22rschmuklerWhile I try and reproduce, here's a stack trace if it helps: https://gist.github.com/rschmukler/8ae09c46ede6bab8d003dc4fe8cccef8#2019-03-0421:22alexmillerwhat sha are you using for tools.deps?#2019-03-0421:22rschmuklere160f184f051f120014244679831a9bccb37c9de#2019-03-0421:23rschmuklerAlso it seems to raise the error w/ about 50% of the deps.#2019-03-0421:23rschmuklerie. I wrote a wrapper that calls add-lib on my project for everything in deps.edn#2019-03-0421:23rschmukler(defn sync-libs
"Dynamically refreshes all libs in `deps.edn` and installs any missing"
([] (sync-libs nil))
([?alias]
(let [edn (-> "deps.edn" slurp read-string)
libs (if ?alias
(get-in edn [:aliases ?alias :extra-deps])
(get edn :deps))]
(doseq [[lib-sym lib-spec] libs]
(try
(add-lib lib-sym lib-spec)
(catch NullPointerException e
(println "Error raised for:" lib-sym lib-spec)))))))
#2019-03-0421:24rschmuklerBut again, this doesn't appear to be an issue in a different project. Still trying to build back up to it happening.#2019-03-0421:28alexmilleradd-lib doesn't re-add any deps it already has btw#2019-03-0421:28rschmuklerThat's fine, I was presuming it to be a no-op#2019-03-0421:29rschmukler(ie. my workflow is I have a file watcher watching deps.edn and it syncs the libs on change. I'm not doing any diffing)#2019-03-0421:29alexmillerthe error seems to imply you can't make a maven system, but not sure why you wouldn't get an error at that point#2019-03-0421:30alexmiller(use 'clojure.tools.deps.alpha.util.maven)
(make-system)
#2019-03-0421:31rschmuklerThat worked (ie. no exception)#2019-03-0421:33alexmillerwell, not really sure. you're doing this under nrepl and it's possible the classloader hierarchy there is having some impact, don't really know.#2019-03-0421:33alexmillerI haven't seen this reported before (although obviously not many people using this)#2019-03-0421:35rschmuklerI'm actually seeing it even if I don't use nrepl (ie. if I just use a plain ol' clojure repl)#2019-03-0421:35alexmillerwell, that's useful in ruling things out#2019-03-0421:35rschmuklerBut yeah, I appreciate that I'm beyond the bleeding edge w/ this. I'll see if I can reproduce it...#2019-03-0421:35alexmilleris there any chance you have this happening in multiple threads a the same time?#2019-03-0421:36rschmuklerI don't think so. I thought it might be some sort of race condition but it seems to be consistent in the dependencies that it fails on#2019-03-0421:36rschmukler(and since it's just a clojure repl, I can't imagine anything spawning threads)#2019-03-0421:38alexmilleryou could try (dotimes [_ 20] (assert (not (nil? (make-session (make-system) default-local-repo)))))#2019-03-0421:39alexmillersee if you get any nils#2019-03-0421:40rschmuklerAlright, will try momentarily#2019-03-0421:43alexmillerI guess would also be good to know which java version you are on#2019-03-0421:45rschmuklerI found the library that causes it#2019-03-0421:46rschmuklerSo, if you have an existing deps.edn w/ metasoarous/oz {:mvn/version "1.5.6"} and then try and invoke add-lib the errors happen#2019-03-0421:46rschmuklerconfirming in my other repo#2019-03-0421:47rschmuklerYep! That does it. https://github.com/rschmukler/clojure-add-lib-exception#2019-03-0421:48rschmuklerNote that the sync-libs is commented out. Digging as to why that lib causes it.#2019-03-0421:50rschmuklerλ java --version
openjdk 11.0.2 2018-10-16
OpenJDK Runtime Environment (build 11.0.2+7)
OpenJDK 64-Bit Server VM (build 11.0.2+7, mixed mode)
#2019-03-0421:53rschmuklerMight be that library's dependency on https://github.com/clojure-goes-fast/lazy-require#2019-03-0421:53rschmuklerYep, this looks like it#2019-03-0421:54rschmukler> Another problem might happen if you mix lazy-require with some other code that does dynamic namespace loading. lazy-require itself makes sure to serialize the loading of namespaces so that nothing breaks because of concurrent loading (e.g., see clojure-emacs/cider#2092). However, if other code also loads namespaces in the background, something might happen. If you ever experience such issues, please file a ticket.#2019-03-0421:55rschmuklerAlthough, it's not like I'm requiring that lib, or even using it. ie. simply having it in the project (oz, not confirmed that it's this library specifically) causes the issue.#2019-03-0421:56alexmillerwell, certainly sounds likely#2019-03-0421:58alexmilleroz is using in it's notebook namespaces#2019-03-0421:58alexmillerwell, I guess specifically https://github.com/metasoarous/oz/blob/master/src/clj/oz/notebook/iclojure.clj#2019-03-0421:59rschmuklerHeh, we are doing exactly the same thing 😛#2019-03-0421:59alexmilleractually that loads it but doesn't use it#2019-03-0421:59rschmuklerAlthough, I don't think it's actually using it. If you look at the lib itself, it's pure#2019-03-0421:59rschmuklerhttps://github.com/clojure-goes-fast/lazy-require/blob/master/src/lazy_require/core.clj#2019-03-0421:59rschmuklerie. it's not like it's doing anything to the run-time or anything#2019-03-0522:21aaelonyI happen to need to use an old (but productionized) box that I’m not allowed to update java on. It’s running java 6. Both lein and clj complain about Exception in thread "main" java.lang.UnsupportedClassVersionError: clojure/main : Unsupported major.minor version 52.0
#2019-03-0522:21aaelonyrunning on linux: #66~precise1-Ubuntu#2019-03-0522:23hiredmanjava 6 isn't support on recent clojure versions if I recall#2019-03-0522:23hiredmanI think 8 is the new minimum#2019-03-0522:24hiredmanhttps://dev.clojure.org/jira/browse/CLJ-2363#2019-03-0522:25hiredmanand the change log for 1.10 says java 8 or above#2019-03-0522:25aaelonythat’s probably what I’m up against. Thanks.#2019-03-0522:26aaelonymaybe there’s an old version of lein I can use…#2019-03-0522:28aaelonyupdate… found a work-around unrelated to clojure… thanks#2019-03-0522:32alexmilleryes, clj and clojure 1.10 both require Java 1.8+#2019-03-0522:32alexmillerI assume "productionized" means "completely insecure" :)#2019-03-0523:37aaelonylol. It means I didn’t have the option of considering any change there… haha#2019-03-0523:43alexmillerYeah, I get it#2019-03-2622:52borkdudeI’m getting a stale classpath for an alias with only one local/root dependency. I added a dependency to the local/root, but now I removed it. Yet clj -A:clj-kondo -Spath still reports it being there.#2019-03-2622:52borkdudeHow/where can I clean this?#2019-03-2622:53borkdude-Sforce did the trick#2019-03-2622:56alexmillerYeah, currently dep changes to local deps do not cause a recalc - need a force for that#2019-03-2622:56alexmillerI have a solution for this but it’s a bit involved#2019-03-2720:19borkdudeis there an install script for MacOS like on linux? using brew on CI takes a bit long#2019-03-2720:36borkdudecaching brew probably also works#2019-03-2720:37hiredmanare you saying your ci server is running osx?#2019-03-2720:37borkdudeyeah#2019-03-2720:37hiredmanwild#2019-03-2720:37borkdudeCircleCI 🙂#2019-03-2720:38borkdudeit’s pretty cool. I can build a binary for linux and MacOS that way#2019-03-2721:04alexmillerYou can use the Linux script on mac I believe#2019-03-2914:20sundbpI'm trying to use a git/url dependency behind a proxy (for an https repo). Any idea how I can set the proxy info that jgit uses under the hood?#2019-03-2914:26alexmillerI think that is an open issue#2019-03-2914:27alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-124#2019-03-2914:27alexmillerhaven't had a chance to look at it yet#2019-03-3020:58sundbpThanks. That’s indeed the relevant issue. bootstrapping the JGit proxy settings to be consistent with either maven settings, or with git settings (https://gist.github.com/evantoli/f8c23a37eb3558ab8765) sounds logical.#2019-04-0110:39mpenetany news on https://dev.clojure.org/jira/browse/TDEPS-51 ?#2019-04-0110:40mpeneta comment says it's fixed but I couldn't confirm it, it still seems broken#2019-04-0111:59mpenetmy bad it does work now#2019-04-0119:41sundbp@alexmiller I'd be willing to do some lifting on tdeps-124 (git proxies) if it is of help and we can agree a direction of where to take proxy info from. It is quite a blocker for my adoption of tools.deps so it is high prio for me.#2019-04-0120:23alexmillergo for it! I think the two main options are to leverage the existing maven info (you can look at how that's accessed in the maven extension) or to look at leveraging whatever is native for git. this probably ends up flowing down into tools.gitlibs too.#2019-04-0120:37sundbpI'm torn between using the git config as more logical, and the cost of 2 config actions required for the user. It is probably still my slight preference.#2019-04-0121:05alexmillermake a list of the tradeoffs#2019-04-0121:05alexmillerin the ticket#2019-04-0219:38sundbp@alexmiller i’m confusing myself. signed up for a JIRA account and was going to comment on that ticket but don’t seem to see any Reply type button anywhere. Is it permissioned?#2019-04-0219:43alexmilleryeah, gimme a sec#2019-04-0219:44alexmillershould work now#2019-04-0220:36sundbpthanks#2019-04-0221:51sundbpput down the 3 ways I’ve been thinking about in that ticket.#2019-04-0513:09Laverne SchrockOpinion:
Of the options presented on the ticket, all except for the "Use maven proxy" seem reasonable. A single source for the proxy config is nice, but Maven settings should only be for fetching things via Maven. If we want centralization I'd vote for the "environment variable" or the "JVM flags" options (though I'm unclear on what the latter would end up looking like).#2019-04-0222:06sundbp@alexmiller another thing I’ve been running into is the monorepo story. I’ve so far adopted the convention to “hijack” CLJ_CONFIG via wrapper tools in my monorepo. E.g. I make it impossible for the user to control CLJ_CONFIG, and use it as an escape hatch to merge in one more deps.edn into the mix. This is obviously not ideal, and leads to discussions like this one in terms of tooling etc: https://github.com/juxt/pack.alpha/pull/34. Similarly to -Sdeps, is there any milage in a similar flag taking an actual additional deps.edn file? e.g. -Sdeps-file which can be given 0 or more times. That’d make at least the parts of the monorepo cases I’ve considered more manageable. Has it already been considered? If not any initial thoughts?#2019-04-0222:07alexmillerinterested in doing more, haven't had a chance to think seriously about it#2019-04-0303:26lilactownAny examples using deps.edn + packr?#2019-04-0313:01sundbpI seem to run into some classloader issue when compiling gen-class using clj.#2019-04-0313:01sundbpI'm trying to use compile with a gen-class. I get a class not found exception when the constructor is defined taking a class from a lib I'm using. I can import that same class, and changing the type to eg String makes compile work. Any caveat about types for constructor args and class path etc?#2019-04-0313:01sundbpThe same setup with lein works OK.#2019-04-0313:03sundbpIt is like the classloader is out of whack. If I start a rebel repl I can import the class etc, and I can compile from the repl. But when passed via the -e flag the same expression gives class not found exception.#2019-04-0313:10sundbpIf I wrap it into something simple that does the same thing via a -m main method it is OK. But the -e version with same code fails.#2019-04-0521:11rafaelI'm seeing an Error building classpath message when running clj. I'm running with -Sverbose but it doesn't help to identify the problematic dependency.#2019-04-0521:13rafael$ clj -Sverbose -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.9.0"} midje {:mvn/version "1.9.7"}}}'
version = 1.9.0.329
install_dir = /nix/store/i2zsj4qknknpfrvfhpx0z69c44b0c12j-clojure-1.9.0.329-prefix
config_dir = /home/rafael/.clojure
config_paths = /nix/store/i2zsj4qknknpfrvfhpx0z69c44b0c12j-clojure-1.9.0.329-prefix/deps.edn /home/rafael/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/2657530762.cp
Refreshing classpath
Error building classpath. In the dependency graph, library org.clojure/clojure specifies version range "[1.2.1],[1.3.0]" - please specify a version in your configuration.
#2019-04-0521:13rafaelShould I open an issue? I'm having a hard time finding out how to proceed debugging the problem.#2019-04-0521:14alexmillerUpdate your clj version#2019-04-0521:15alexmillerSome older versions like this one didn’t handle version ranges#2019-04-0521:15alexmillerNewer ones do#2019-04-0521:17rafaelNewbie question: is there a way to test a new clj version without updating the binary installed in the system?#2019-04-0521:27lilactownI think it's literally a shell script, you could try downloading it and executing it in a local dir?#2019-04-0521:28alexmillerIt’s both a script and an uberjar#2019-04-0521:29alexmillerThey need to be matched up#2019-04-0521:31alexmillerThere isn’t an easy way to do what you’re asking but you could always rollback to the version you’re on - it’s just the same url with the version you want#2019-04-0521:31rafaelThanks!#2019-04-0813:15stathissiderisdeps has made such a big difference to my Clojure workflow! I’m eternally grateful for Leinignen for taking us this far, but tools-deps has made things even better#2019-04-0813:21alexmillerthx!#2019-04-0814:31stathissiderisI’ll list some specific benefits that I’ve seen:
1. Less mono-repo oriented development due to effortless git deps (this was huge for me).
2. I’m more likely to fork open source libraries (and attempt to fix them) also because of git deps.
3. Casually adding custom automated project-related “tasks” (DB migrations, linting, testing etc) because of the combination of aliases and Clojure “scripts” (where with lein I would need a plugin)
4. Stopped using project generation tooling, I now make a deps.edn, an src folder and I suddenly have a new Clojure project!#2019-04-0814:39kirill.salykin> 3.
cant you do same with lein alias?#2019-04-0814:49souenzzoyeah, but lein culture says to develop a plugin.#2019-04-0815:05stathissideristo the point where I was not aware of lein alias 😄#2019-04-0815:12stathissiderisoh I did know it, but it requires a main method#2019-04-0916:15borkdudehow can I add compiled .class files to the classpath with tools.deps?#2019-04-0916:17dominicmPut them in a folder, add it to your paths.#2019-04-0916:17borkdudeso I put it in target/classes and then :paths ["src" "target/classes"]?#2019-04-0916:18alexmilleryep#2019-04-0916:18borkdudeI tried this from the command line: -J-cp target/classes but that didn’t work#2019-04-0916:18alexmillerno, that won't work as clj is building the classpath#2019-04-0916:19alexmillerprobably should actually make that an error#2019-04-0916:19borkdudeand if target/classes doesn’t exist, it won’t bail out?#2019-04-0916:19alexmillerthat's the one java property you can't set :)#2019-04-0916:19alexmillermissing dirs are fine#2019-04-0916:19borkdudecool#2019-04-0916:19alexmillerjava doesn't care#2019-04-0916:19alexmiller(but it also won't pick up stuff that gets added later, I don't think)#2019-04-0916:19borkdudethat’s fine#2019-04-0916:20borkdudeI only need to compile a class if I’m using GraalVM compilation, for normal JVM usage it’s unnecessary#2019-04-0916:20alexmilleryeah, should be fine#2019-04-0916:20borkdudeso this dir is going to be in my gitignore#2019-04-0916:20alexmillermakes sense#2019-04-0917:16rschmuklerIs it possible to set environment variables via the deps.edn file, similar to the :env section of a project.clj? If not, what's the recommended way to handle that when using deps.edn?#2019-04-0917:19hiredman:env as a way to set env variables is something some lein plugin provides#2019-04-0917:20hiredmanI don't think that plugin actually sets environment variables, it provides its own kind of shadow system that you get access to as long as you use that plugin's functions for reading env variables#2019-04-0917:21rschmuklerAh! I misunderstood. There was a github issue that made it seem like it did something better than that -namely they were overwriting LD_PRELOAD and it was working it seemed#2019-04-0917:22rschmukler(I'm hooking into some C FFI with an old relic of a library)#2019-04-0917:23hiredmanit looks like environ dumps the environment stuff to .lein-env, which is maybe a standard thing? I am not sure#2019-04-0917:23hiredmanbut like, if you have environment variables, why not just set them?#2019-04-0917:24hiredmanLD_PRELOAD=... clj ...#2019-04-0917:24rschmuklerI don't want to overwrite them outside of the project, and I typically invoke clojure via cider#2019-04-0917:24rschmuklerso, adding project specific environment variables to my emacs config feels quite dirty#2019-04-0917:24rschmukler(and requires all engineers on the project solve it on their own)#2019-04-0917:33rschmuklerI was able to solve it via dir-locals w/ emacs, for anyone interested...#2019-04-0917:33rschmukler((nil . ((eval . (setenv "LD_PRELOAD" "libcurl.so.3")))))
#2019-04-0917:55alexmilleron clj command line, you can use prefix -J to pass env vars - like -J-Dmy.prop=val#2019-04-0917:56alexmilleror you can put those in an alias in deps.edn under :jvm-opts#2019-04-0917:56alexmiller{:aliases
{:foo {:jvm-opts ["-Dmy.prop=val"]}}}#2019-04-0917:57alexmillerclj -A:foo#2019-04-0917:57rschmuklerIs there a default equivalent?#2019-04-0917:57alexmillerno, but that has been requested multiple times and I expect will be added when I next get around to it#2019-04-0917:57alexmillerthere's a ticket for it#2019-04-0917:58rschmuklerCool cool. It's completely understandable that there wouldn't be one (yet, or even ever). Thanks for the response!#2019-04-0917:58alexmilleroh, you're asking about env vars#2019-04-0917:59alexmillerI'm talking about jvm opts#2019-04-0917:59alexmillerthose are different things#2019-04-0917:59alexmillersorry for the confusion#2019-04-0917:59alexmillerI consider that out of scope for clj right now#2019-04-0918:00rschmuklerMakes complete sense. Frankly, I think it probably introduces more nasty things than it's worth (eg. people using it as a config store for default environment variables, etc)... This is definitely an edge-case#2019-04-0918:00alexmilleryeah, I'll resist adding that as long as I can :)#2019-04-0918:00rschmukler👍 I think that's a good call#2019-04-0918:02seancorfieldThat begs an interesting question: when folks are providing "environment" to processes, do they a) use only environment variables b) use only JVM properties or c) use a combination with one overriding the other? And, for c) which overrides which?#2019-04-0918:04rschmukler@seancorfield As someone who came to Clojure (and the JVM) after heavily integrating docker + kubernetes, I can tell you I use environment variables for almost everything, and, if possible, don't intend to ever use or learn JVM properties#2019-04-0918:06rschmuklerI think it's mostly that containers (and golang especially) harbored in a lot of workflow / tooling around the idea of using environment variables, and it's something that can apply to outside any one run-time. ie. I can write a service and guarantee environment vars as a config option... If I switch away from JVM for any reason, I now need to provide a config migration story too#2019-04-0918:07rschmuklerregarding (c) - I would expect environment variables to be the last authority... but I think that's likely because of the esteem that I hold them in above#2019-04-0918:09alexmillerusually in an app I load config from an edn file, but allow override via env, and don't use jvm props at all#2019-04-0918:10alexmillerI find that gives you a lot of flexibility and options for local dev, CI, prod#2019-04-0918:10alexmillerbut then I don't work on many apps :)#2019-04-0918:11rschmukler@alexmiller that's exactly what I've done as well. I will sometimes have multiple configs corresponding to an environment (`prod.edn`, dev.edn, test.edn) which might overwrite certain keys in config.edn. I then use ENV to determine which env-specific config to use, while still allowing overwrites for specific vars via environment variables#2019-04-0918:12rschmuklerElixir also does something similar to the above#2019-04-0918:14alexmillerit's not well-known, but I like many things about https://github.com/levand/immuconf#2019-04-0918:16alexmillerif you're willing to ignore the ENV stuff above and use only a stack of appropriately created config files#2019-04-0918:17rschmuklerBrilliant! Similarly, duct does EDN-based config quite well. The docs for it are a bit lacking... But they let you do coercion and environment variables directly via EDN reader macros#2019-04-0918:17rschmuklerhttps://github.com/duct-framework/core/blob/master/src/duct/core/env.clj#2019-04-0918:18alexmilleryeah, I like duct too from what I recall#2019-04-0918:18alexmillerI've used environ quite a bit, but can't say I love it#2019-04-0918:20rschmuklerDuct + Integrant have quickly become my favorite way to build / prototype applications. I feel like I'm building an arsenal of components that I can use to build almost any system that I have seen in my career. They integrate the config map w/ spec so you get good error messages, while still having a single map that shows the entire system. It's basically stuartsierra's component extended to a map / graph, rather than relying on manual instantiation from the programmer#2019-04-0918:20alexmilleryeah, weavejester is great at maximizing the Clojure Way ™#2019-04-0918:21rschmuklertrade mark#2019-04-0918:21rschmukler😉#2019-04-0920:01jcfHi all. I'm looking into deploying a small toy project to Heroku inside a Docker container and I'm running into an interesting problem with my use of tools.deps.
When I build the Docker container locally I can see all the dependencies being downloaded. When I push the container to Heroku and run the thing the same dependencies are installed all over again, which takes so long Heroku kill my container and the app fails to start.
Is there anything container-specific to look out for with tools.deps? My first guess was there was a missing Maven repo somewhere but when I run the container locally all the dependencies are there. Naturally I'm now wondering if Heroku do something different on their platform that might prevent tools.deps from noticing the Maven repo.#2019-04-0920:03jcfI'm running clojure -Srepro -e '(println "Dependencies installed.")' to install things in my Dockerfile, and then I run the app with exec clojure -Srepro -A:measure -m foo.bar.main. The measure alias only adds a single JVM opt.#2019-04-0920:03jcf:measure {:jvm-opts ["-Dio.pedestal.log.defaultMetricsRecorder=foo.bar.report/reporter"]}
#2019-04-0920:05jcfI can reproduce the artefact-download behaviour if I run the container on Docker and invoke my CMD by hand.#2019-04-0920:05dominicmIs it the same user?#2019-04-0920:06jcfAnd I think I might have cracked it. It looks like ~/.m2 is /app/.m2/repository when I run the container but the Docker build step will install things in /root/.m2.#2019-04-0920:06jcf@dominicm I think you've cracked it too!#2019-04-0920:07jcfI'm guessing Heroku run things as some other user, which messes with $HOME. I wonder if I can tell tools.deps where to put the Maven repo with the Docker invocations only. Don't really want to mess with that in the project deps.edn as it's kinda up to the dev where they want to keep Maven artefacts.#2019-04-0920:08jcfDoes :maven/local-repo not work in an alias? 🙂#2019-04-0920:09dominicm@jcf try -Sdeps perhaps? But remember permissions.#2019-04-0920:10jcfI'll give that a stab.#2019-04-0920:20jcfI'm giving -Dmaven.local.repo=/app/.m2 a try. Let's see if that does the trick. #docker 🙂#2019-04-0920:24jcfThat didn't work. Because it's maven.repo.local. 😕#2019-04-0920:24alexmilleryou can set it in deps.edn#2019-04-0920:24alexmiller:mvn/local-repo "/app/m2" as a root level entry#2019-04-0920:24alexmilleror whatever you need#2019-04-0920:25jcf@alexmiller I was hoping to only specify that for the Docker side of things to enable the default behaviour for local development.#2019-04-0920:25alexmillerthat's not something you can set in an alias#2019-04-0920:25alexmillerbut you could append by using clj -Sdeps '{:mvn/local-repo "..."}'#2019-04-0920:26dominicmThat's what I was trying to say ^^ but with the brevity that mobile prefers :)#2019-04-0920:26jcfI tried that originally and the running container still installed everything. I'll bring back the -Sdeps ... approach and poke around inside the container to see if the user in the running container maybe can't access the :mvn/local-repo.#2019-04-0920:39jcfThe -Sdeps {:mvn/local-repo "/app/.m2"} did the trick! Just had to blow away the container on Heroku and push it up again.
Now I need to make sure we stick within allocated memory and start up fast enough despite not installing deps. Might have to uberjar yet.
Thanks for the help, @dominicm @alexmiller!#2019-04-0921:09alexmillernp#2019-04-1106:17borkdudeif I check some files into a resources folder in my git repo, will someone who uses my repo as a git dep be able to see these resources? I remember there was an issue with this in tools.deps?#2019-04-1106:17borkdudeor was this only when you’re using it as a local/root?#2019-04-1106:25borkdudeafter some testing, both seem to work#2019-04-1106:25borkdudeI remember what the issue was with this. maybe a caching issue#2019-04-1115:00borkdudewith leiningen it seems that lib/tools.jar is automatically on the classpath. with tools.deps it doesn’t seem so. what’s the best way to do it without using a cmd line like:
clj -Sdeps "{:deps {tools.jar {:local/root \"$JAVA_HOME/lib/tools.jar\"}}}"
#2019-04-1115:12alexmilleryou should not have to do this#2019-04-1115:13alexmillerit should automatically be included if you are using the JDK (not the JRE) afaik#2019-04-1115:13alexmillerare you accidentally using the JRE?#2019-04-1115:16alexmillermight also be some differences from Java 8 vs Java 9+ (post-module system) in the jdk file structure. I'm not sure there even is a tools.jar anymore?#2019-04-1115:18alexmilleroh yeah, as of java 9, tools.jar does not exist anymore#2019-04-1115:19alexmillerneeding to include it directly is weird, and probably wrong#2019-04-1115:30ghadiif the original question was about resources, as long as that directory is listed in the :paths on the dependent repo, then the depending repo will be able to see it#2019-04-1115:31borkdudeI’m using java 8#2019-04-1115:31borkdudea quick test if it works in any of your REPLs:
(import 'com.sun.tools.javadoc.Main)
#2019-04-1115:32borkdude$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home
#2019-04-1115:32alexmillerthat stuff is dead#2019-04-1115:32borkdudeis there something newer I should use?#2019-04-1115:33alexmilleryes - javax.tools.DocumentationTool#2019-04-1115:34alexmillerI don't know the coords for it, but you should just be using it as normal Maven artifact#2019-04-1115:34borkdudethat seems to work out of the box:
$ clj
Clojure 1.10.0
user=> (import 'javax.tools.DocumentationTool)
javax.tools.DocumentationTool
#2019-04-1115:36alexmillerprob doesn't in newer jdks#2019-04-1115:37alexmilleractually, it does on Java 11#2019-04-1115:37borkdudeso I can do something like this as with the older Doclet API? I want to generate EDN#2019-04-1115:38alexmillerI believe it is the replacement for all that old sun stuff, yes#2019-04-1115:38alexmillerI'm sure there are buckets of docs to read about it, I've never used it#2019-04-1115:38borkdudethanks#2019-04-1116:03borkdudeIt seems I do need the “dead” Doclet class if I’m using Java 8, since the jdk.javadoc stuff is only on Java 9#2019-04-1116:24borkdudeso I’m seeing this in the REPL:
(System/getProperty "java.home")
"/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home/jre"
and the same with leiningen. But tools.jar is in ../jdk/lib/tools.jar.#2019-04-1116:27borkdudefunny, it seems in 2012 people wrote a special plugin for this:
https://github.com/pallet/lein-jdk-tools/blob/develop/src/lein_jdk_tools/plugin.clj#L11#2019-04-1116:44borkdudeI wonder that if I’m going to migrate to JDK 11 if my code will still be able to run for Java 8 users if I’m going to use those new classes, so I’m hesitant about it#2019-04-1116:54alexmillerit should be fine. that "new" doclet stuff has been around since at least java 7#2019-04-1116:55alexmillerseems better than building on something that has been pronounced dead and unsupported#2019-04-1116:56borkdudebut the new doclet stuff is only there since Java 9, or maybe I’m not finding the right stuff on google…#2019-04-1116:56borkdudeo wait: https://docs.oracle.com/javase/8/docs/jdk/api/javadoc/doclet/com/sun/javadoc/Doclet.html#2019-04-1116:57borkdudeOK yeah, I’m going to use that.#2019-04-1117:02dominicmWhat are you using doclet for?#2019-04-1117:07dominicmOh, it's for reading docs in java. I see.#2019-04-1117:07dominicmI thought it was for generating them.#2019-04-1117:05alexmillereverything com.sun is on the removal path#2019-04-1117:07ghadihttps://docs.oracle.com/en/java/javase/11/docs/api/jdk.javadoc/jdk/javadoc/doclet/package-summary.html#2019-04-1117:07ghadimake sure you read the migration parts of ^#2019-04-1117:08ghadi@borkdude#2019-04-1117:43borkdude@ghadi thanks!#2019-04-1117:44borkdudeOld type: SeeTag
New type: com.sun.source.doctree.LinkTree
com.sun.source.doctree.SeeTree
Hmm..#2019-04-1117:46borkdudeCrap, it does seem that this new stuff is only available since 9#2019-04-1117:47borkdudeAnyway, thanks for the help.#2019-04-1117:51nopromptDo we have tool for releasing to Clojars in deps land?#2019-04-1117:52borkdudeThere are a bunch of these tools on the wiki#2019-04-1117:52nopromptI’m still lein deploy clojars. 😐#2019-04-1117:52borkdudeMe too.#2019-04-1117:52nopromptI reckon if any of those were as convenient as that you’d have mentioned it. 🙂#2019-04-1117:53borkdudeHaven’t tried them yet#2019-04-1117:54nopromptI just want a microwave.#2019-04-1117:54nopromptA jar cooking microwave.#2019-04-1117:55borkdudedepstar is one that I’ve heard of#2019-04-1117:55alexmilleryou can call lein fns from clj#2019-04-1117:55borkdudehttps://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging#2019-04-1117:57borkdude@noprompt this is an example of calling lein using tools.deps: https://github.com/oakes/full-stack-clj-example
nice hack#2019-04-1117:59alexmillerI added that to the wiki page :)#2019-04-1119:28mpenetit doesnt work with git deps, it uses lein for dependency resolution when building the uberjar https://github.com/oakes/full-stack-clj-example/blob/master/prod.clj#2019-04-1119:28mpenetI'd still favor pack/depstar#2019-04-1119:29mpenetI guess it could be improved, the concept is quite interesting#2019-04-1117:58nopromptCool. I’ll check that out. pack looks pretty nice. The Juxt folks tend to make decent things.#2019-04-1120:50dominicmI appreciate the complement. Let me know if you have any issues.#2019-04-1209:55sooheonDoes the capsule method of uberjaring not support AOT compilation?#2019-04-1210:06dominicmIt works with AOT'd assets#2019-04-1210:06dominicmWhy do you think it doesn't?#2019-04-1210:10sooheonI’m coming from lein uberjar where you explicitly give it a ns to AOT compile. Also saw that Capsule main didn’t care about :gen-class. Do you mean I should AOT compile with something else, and provide the output of that as extra-paths?#2019-04-1210:11dominicmIf you really want to AOT, yeah, I just use clojure -e#2019-04-1210:12sooheonOk thanks#2019-04-1117:58borkdude(it’s not an off the shelve solution)#2019-04-1118:13martinklepschI found that there's a thing for every step of the process but none that puts all those bits together sadly#2019-04-1118:50seancorfieldWe're using (my fork of) depstar at work for building uberjars. I'm also using it to build "thin" jars for deploying to Clojars (and I use mvn deploy for that, so I run clj -Spom, clj -A:uberjar (alias for depstar's uberjar entry point), and then mvn deploy:deploy-file ...)#2019-04-1118:50ghadi^ @noprompt#2019-04-1118:50ghadithat will get you going#2019-04-1118:51seancorfield(well, I have a little shell script that combines that -- and also updated the pom.xml with the current SHA of the project so http://clojars.org can point to the commit on GitHub)#2019-04-1118:53nopromptNice. I’ll give this a try.#2019-04-1119:22borkdudeDarn.
> This class must contain the start(Root) method.
This is in Java 11 documentation for javadoc which is also used by those new ToolProvider thingies.
But the start(Root) method is from the ancient com.sun code. So this buys me nothing… The documentation for the new Doclet interface doesn’t even mention this though.
https://docs.oracle.com/en/java/javase/11/tools/javadoc.html#2019-04-1200:42cflemingIs anyone using deps with polyglot JVM projects, e.g. containing Java/Kotlin/etc?#2019-04-1200:42cflemingI’m interested in recommendations for approaches if so.#2019-04-1200:43cflemingI see there are some lightweight build tool options listed in the wiki but I don’t know what the state of any of them is.#2019-04-1201:20alexmillerI probably wouldn’t use deps as the controller for that, I’d use it under make, or whatever#2019-04-1211:52sundbpCan’t recall if I brought this up before, but can’t find any traces (tickets etc). So we have -Sdeps to add “extra stuff”. For my purposes it’d be very useful to have the equivalent to add extra deps.edn files to be merged in, e.g. -Sdepsfile path/to/extra/deps.edn. As it is logically the same as -Sdeps "$(< path/to/extra/deps.edn)", just packaged in a different way - would there be any pushback to adding such an option? @alexmiller#2019-04-1212:42alexmillerThere is a ticket for this and I’m not opposed to it, but am still thinking about various aspects#2019-04-1212:56orestisWait, -Sdeps adds extra stuff? Can it be combined with -A then?#2019-04-1213:46dominicmIt can, yeah.#2019-04-1213:47dominicmTo make sure I don't clobber things, I use a namespaced alias name in scripts, sometimes with generated extra paths (e.g. In tmp)#2019-04-1213:47alexmiller-Sdeps adds an additional "deps.edn" on the end of the merge chain#2019-04-1214:16orestisOhhh that’s so nice. #2019-04-1219:49sundbp@alexmiller great to hear.#2019-04-1308:19borkdudeif I make an alias, I should be getting the extra paths of that alias when doing -R:the-alias right?#2019-04-1308:19borkdudesomehow it’s not working for me, but it is working with the -A option#2019-04-1308:22borkdudeoh, -C… 🙂#2019-04-1520:35borkdudeis there a way to let tools.deps print only the paths that you specified in your deps.edn file and not the entire classpath?#2019-04-1520:37alexmillerNo - why would that be useful?#2019-04-1520:37borkdudelinting 🙂#2019-04-1520:38alexmillerWhy?#2019-04-1520:38borkdudewhy linting you mean?#2019-04-1520:40borkdudeone use case: I want to lint the entire classpath, but only report errors from my own code. but it’s not too hard to do some manipulation on the output of -Spath or hard-code it. the amount of paths if fairly limited.#2019-04-1520:45borkdudeto answer your question more generally: it could in theory be useful to list all the files from your own paths for whatever reason#2019-04-1520:45borkdudebut I guess not high priority for now#2019-04-1605:54orestisShouldn’t be that complicated to read the file and get the paths out, right?#2019-04-1711:54orestisIs there an invocation of Clojure that will give the current version number?#2019-04-1711:56mpenet*clojure-version* I think#2019-04-1711:56mpenet*clojure-version*
{:major 1, :minor 10, :incremental 0, :qualifier nil}
#2019-04-1712:03orestisAh, sorry - I meant about the Clojure CLI tools#2019-04-1712:04orestisWhen installing in Linux, I don’t want to download curl -O every time if it’s already present and at the correct version.#2019-04-1712:08dominicmclojure -Sdescribe?#2019-04-1712:10dominicm❯ clojure -Sdescribe | grep -o ':version ".*"' | sed 's/:version //' | sed 's/"//g'#2019-04-1712:14orestisFantastic! Thanks 🙂#2019-04-1712:38orestisOK, I’m having trouble running the Clojure CLI tools in Elastic Beanstalk, and the AWS support this time isn’t very helpful.#2019-04-1712:38orestisIn my logs, I see this error: mkdir: cannot create directory '/.clojure': Permission denied#2019-04-1712:39orestisWhich indicates to me that something is trying create a .clojure directory at the root of the filesystem, which is weird.#2019-04-1712:42dominicmAre you installing during user data?#2019-04-1712:42dominicmDoes -Sdescribe try and create .clojure unconditionally?#2019-04-1712:48orestisIt seems so — it’s a mess because Elastic Beanstalk is just duct tape under the hood.#2019-04-1712:48orestisSo you run your Java command with ruby foreman and supervisord…#2019-04-1712:50orestisGot it, they were setting HOME=/tmp whereas the user actually running the clojure command didn’t have permissions there.#2019-04-1712:50orestisSetting CLJ_CONFIG to /home/webapp/.clojure seemed to have fixed the problem.#2019-04-1712:59alexmillerthere's a ticket about working in read-only fs btw#2019-04-1712:59alexmillerhttps://dev.clojure.org/jira/browse/TDEPS-119#2019-04-1713:08orestis.ebextensions/setup.config file that works:#2019-04-1713:08orestisoption_settings:
- option_name: CLJ_CONFIG
value: /home/webapp/.clojure
commands:
01_install_clojure:
command: |
curl -O
chmod +x linux-install-1.10.0.442.sh
sudo ./linux-install-1.10.0.442.sh
#2019-04-1713:12alexmillercurrently, clj does need some writable path to work as it always drives at least the cp through the cached cp file#2019-04-1713:17orestisPosted a quick guide to https://clojureverse.org/t/running-a-clojure-application-on-aws-elastic-beanstalk-with-clojure-cli-tools/4161#2019-04-1713:17orestisThe project .cpcache works fine, it’s the $CLJ_CONFIG that was giving me trouble…#2019-04-1718:19tanzoniteblackis there an equivalent of lein's :init-nsconfig, where you specify which namespace to load and switch into by default when starting a repl?#2019-04-1718:37dominicm-e "(in-ns 'foo)"#2019-04-1721:15kennyAny ideas why I'd hit this? I'm running on a CI that has permission to git clone via HTTPS.
Error building classpath. ...: Authentication is required but no CredentialsProvider has been registered
#2019-04-1721:17kennyShoot. Looking at tools.gitlibs, it seems like it may not support cloning private repos via HTTPS 😞#2019-04-1721:22ghadiIt doesn't. I've worked on that.#2019-04-1721:22ghadiHave two different approaches for tackling that.#2019-04-1721:25kennyAh, I see TDEPS-91 and TDEPS-33. I assume there's no workaround for this right now?#2019-04-1721:27kennyI suppose I could try using the shell-git branch here: https://github.com/ghadishayban/tools.gitlibs/tree/shell-git.#2019-04-1721:27kenny... if you could override the tools-deps version the CLI uses.#2019-04-1723:22ghadi@kenny GitHub or otherwise?#2019-04-1723:23kennyWhere my git deps are coming from?#2019-04-1723:39alexmilleryou can override it if you just hack your clojure script :)#2019-04-1723:40alexmillerjust prepend the source dir to tools_cp#2019-04-1800:00kennyInteresting... Thinking I'll attempt the path of adding an SSH key to the CI. If too painful, will try the clojure script hacking.#2019-04-1808:34jeroenvandijk@kenny You can also play with gitconfig and for instance switch between ssh and https without code changes # [url ""]
# insteadOf =
#2019-04-1808:34jeroenvandijkpart of my ~/.gitconfig#2019-04-1814:42kenny@jeroenvandijk But this won't let tools-deps support https auth, correct?#2019-04-1814:44jeroenvandijkMy guess is that you can even achieve that https://git-scm.com/docs/gitcredentials#2019-04-1814:45jeroenvandijki did only the opposite as the https auth for github is limited#2019-04-1814:45jeroenvandijkwe're using circleci and they even have a feature to add an ssh key of another user (over a deploy key) to the project automatically#2019-04-1819:09dominicm@alexmiller what do you think about gitlibs making clones read only so that they aren't accidentally modified by user error?#2019-04-1819:11alexmilleryou mean the working tree I presume#2019-04-1819:11alexmillerseems reasonable#2019-04-1819:12ghadithere's no such thing#2019-04-1819:13ghadireadonly directory means you cannot add or remove to it. modifying an existing file is still fair play#2019-04-1819:13ghadiyou'd have to ensure that every file was r/o too#2019-04-1819:14alexmillerI assumed that's what we were talking about#2019-04-1819:14dominicmI'll lodge a JIRA#2019-04-1819:15alexmillerthe dirs under ~/.gitlibs/libs are sha-pegged and shouldn't change#2019-04-1819:16alexmillerdoes changing those properties affect whether it looks like the correct working tree to git?#2019-04-1819:17ghadihuh?#2019-04-1819:19alexmillerI guess I'm asking if file attributes affect the sha, but looks like no#2019-04-1822:00dominicmSome do, like +x#2019-04-2508:56jeroenvandijkWould it be a good idea to add the :inclusions to a maven entry? I'm updating the aws-java-sdk library and i'm really only interested in a few of it's dependencies, but it is exploding#2019-04-2510:00delaguardoyou have :exclusions key defined for maven entry. And it has much less abstract (or opinionated) meaning than :inclusions#2019-04-2510:06jeroenvandijkYep. So :exclusions would become unreasonably big with aws-java-sk therefore whitelisting seems like a better approach. Especially since an update means new exclusions#2019-04-2510:09delaguardoas far as I understand aws-java-sdk is just a collection of packages grouped bu functionality. If you are interested only in one particular package - add just this package and do not add aws-java-sdk#2019-04-2510:11delaguardolike this {com.amazonaws/aws-java-sdk-s3 {:mvn/version "1.11.539"}}#2019-04-2510:40jeroenvandijkOk i guess you have never experienced this problem. I have#2019-04-2510:40jeroenvandijkThanks for trying to help though#2019-04-2511:06delaguardoYou might be surprised) maybe you can describe you pain more? it could be helpful for anyone how is willing to help you#2019-04-2511:20jeroenvandijke.g. when I try to use Amazonica (`[amazonica "0.3.142"]`) for specific resources I would need to know what to exclude whereas I actually know what to include if I just want to do some s3 things#2019-04-2511:46delaguardoOk, so consider this https://github.com/mcohen01/amazonica#for-the-memory-constrained
I also made a gist with dependency trees for two cases: amazonica without any classpath changes and amazonica with just s3 added
https://gist.github.com/DeLaGuardo/4e32e950fa7ee7b594c9df2e97a10fd4#2019-04-2511:46delaguardoMain idea - exclude everything, then add parts you know#2019-04-2511:59delaguardoand btw - there is a bit nicer library to work with aws s3 - https://github.com/cognitect-labs/aws-api#2019-04-2512:08jeroenvandijk> Main idea - exclude everything, then add parts you know
Yeah I think you are right. Maybe I was just being impatient#2019-04-2512:31alexmillerThere are split out versions of the stuff under aws-java-sdk#2019-04-2512:35alexmillerSee https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/setup-project-maven.html#2019-04-2512:42jeroenvandijkthank you, i guess i was just being impatient.#2019-04-2514:30unbalancedalright... trying to fork/repair an out dated library (https://github.com/benashford/redis-async) which includes some local Java classes#2019-04-2514:31unbalancedso I converted the project.clj to deps.edn for starters#2019-04-2514:32unbalanced#2019-04-2514:32unbalancedthe directory structure is as follows:#2019-04-2514:32unbalanced#2019-04-2514:33unbalancedthe problem comes when running the tests, the Java classes do not seem to be importing properly#2019-04-2514:33unbalanced#2019-04-2514:34ghadi.java files need to be compiled into .class files#2019-04-2514:34ghadithey cannot be used raw#2019-04-2514:34unbalancedahh interesting#2019-04-2514:34ghadiI would suggest separating the java part of the project into a different artifact, and then including that in the clojure deps.edn project#2019-04-2514:35unbalancedI think that's a good suggestion. Do you include it with {:local/root "../local-path"} as one would with a clojure project?#2019-04-2514:36unbalancedor is there a different way to include local java?#2019-04-2514:36unbalancedor should I upload it to maven or something?#2019-04-2514:36ghadiupload to maven or clojars#2019-04-2514:37unbalancedyou can upload java to clojars?#2019-04-2514:37ghadii think so#2019-04-2514:37ghadi"it's just a jar"#2019-04-2514:37unbalancedcool... will look into it. Ah true story#2019-04-2515:08unbalancedare you sure you can't use .java files without compiling first...? I thought there was some kind of implicit compilation as long as class path was set up properly#2019-04-2515:08unbalancedbecause it worked fine for leiningen#2019-04-2515:08unbalancedvia the key :java-source-paths ["jresp/src/main/java"]#2019-04-2515:10ghadiyeah, I'm sure#2019-04-2515:11ghadilein calls javac#2019-04-2515:15unbalancedah so that's fundamentally different for the deps.edn approach#2019-04-2515:54alexmilleryes#2019-04-2516:33seancorfieldYou can always use the -e option to shell out to a call to javac 🙂#2019-04-2516:33seancorfield(no, don't, it's horrible... but it does work 🙂 )#2019-04-3009:33yuhanIs there any way to use a lein plugin in a tools.deps project? E.g. Codox, lein-ns-dep-graph#2019-04-3009:57delaguardo@qythium most of lein plugins is just a normal projects. You can include them in your dependencies list and execute their -main function if exists or make your own custom -main. E.g. https://github.com/weavejester/codox/blob/master/codox/src/codox/main.clj#L116 there is main function to generate codox documentation. It requires some arguments that you have to assemble by yourself but there is nothing impossible.
clj -Sdeps '{:deps {codox {:mvn/version "0.10.6"}}}' -e "(do (require 'codox.main) (codox.main/generate-docs))"
Oneliner as a prove of concept#2019-04-3016:50seancorfieldAnd remember, if you're on Clojure 1.10, you can use requiring-resolve so that becomes -e "((requiring-resolve 'codox.main/generate-docs))"#2019-04-3010:08yuhanwow, I can't believe that just worked out of the box 😮#2019-04-3010:10yuhanfully formatted HTML docs of all the namespaces in a large project with zero setup, how awesome is that#2019-04-3010:10yuhanthanks!#2019-04-3010:32Christian Johansenwhen changing the sha on git deps, I frequently run into a variation of this error:#2019-04-3010:32Christian JohansenCloning:
#2019-04-3010:32Christian Johansenis this a known problem, or should I report it in jira?#2019-04-3010:34dominicmI'm not aware of any similar reports in this channel, nor have I encountered it itself.#2019-04-3010:34Christian Johansenhmm, I wonder if it could be a false message. I removed the lib from ~/.gitlibs/_repos, and then I see that it errors on auth, as I haven't sun ssh-add after a recent reboot#2019-04-3010:34Christian Johansenso maybe that error above also was really caused by an authentication failure?#2019-04-3010:41dominicmMaybe. Could be that there's more to be done with catching it correctly too though.#2019-04-3011:43alexmillerI haven’t seen anything like this from anyone#2019-04-3013:02Christian Johansenok. I'll see if I can reproduce it and if so, report#2019-05-0107:17luposlipdeps.edn newbie here: Anyone's got some intelligent input to the following issue, where a working lein based project, using a java library hosted on jitpack, doesn't work when used via deps.edn (dependency tree doesn't get resolved)?
https://github.com/luposlip/json-schema/issues/1#2019-05-0107:44dominicm@luposlip you need to put jitpack in your :mvn/repos#2019-05-0107:47luposlipeven when it's already referred to by the json-schema library?
The user wants to use json-schema, which already refers to jitpack, from a project using deps.edn.#2019-05-0107:59dominicmI'm not sure that deps can pick those up, there might be a jira.#2019-05-0108:01dominicmhttps://dev.clojure.org/jira/browse/TDEPS-46#2019-05-0108:01luposlipHmm.. When a library is packaged and deployed to clojars it normally works out of the box when using it from another lein project, regardless of the dependencies and where they are stored.
If this is not the case with clj + deps.edn it seems like a showstopper to me.#2019-05-0108:01luposlipAhh, thanks for the link!#2019-05-0108:02luposlipI'll follow that jira! 🙂#2019-05-0108:02dominicmYour vote will help with setting priority#2019-05-0108:03luposlipI will upvote it!#2019-05-0123:02kszabohey, moving from boot to tools-deps + kaocha, which of the many uberjar building tools do you recommend to replace boot’s capabilities? I see several but see no consensus from the community on which is the most stable / can expect future maintenance.#2019-05-0123:02kszabohey, moving from boot to tools-deps + kaocha, which of the many uberjar building tools do you recommend to replace boot’s capabilities? I see several but see no consensus from the community on which is the most stable / can expect future maintenance.#2019-05-0123:21kennyWe've been using https://github.com/seancorfield/depstar in production for several months now. Does the job.#2019-05-0123:43kszabothanks, but I just noticed that it doesn’t do AOT, guess I’ll need to add that to the list of required features so that the build tooling change does not change the produced artifact#2019-05-0123:56kszaboIt seems badigeon will fit our needs, thanks!#2019-05-0123:58seancorfieldAOT is evil 🙂#2019-05-0200:04kennyYep - we stopped AOT'ing our code when we moved over from Boot. Too many headaches.#2019-05-0200:07seancorfieldWe stopped AOT'ing long before that 🙂 Hence depstar does exactly as much as we need: build a JAR from the classpath. In production we do java -cp /path/to/the.jar clojure.main -m our.entry.point#2019-05-0201:01donaldballWhat is the current reason to consider AOT evil? We’ve just started using it for our service uberjars to shave a modest but meaningful time off of startup time.#2019-05-0203:00seancorfieldIf you're using it as the very last stage, to produce an uberjar, and you really need it to improve startup time, that's "fine". But the Leiningen default where the main namespace is :gen-class'd and AOT is the default in project.clj just gets beginnners into a mess -- and it is completely unnecessary. AOT overall is completely unnecessary -- unless you really, really care about startup time for a final production artifact.#2019-05-0203:01seancorfieldAOT's transitive compilation issues and the problems you can run into when you end up with a mix of .class and .clj versions of the same file can be hell to debug. Much better to just avoid all of that, unless you know what you're doing and you see AOT as purely an end-of-the-line optimization.#2019-05-0203:01seancorfieldThe biggest overhead for us at startup time is New Relic's Java agent instrumenting classes (since Clojure generates so many).#2019-05-0203:02seancorfieldAnd since nearly all of our processes are long-running, we just don't care about startup time.#2019-05-0215:03kennyI could see it mattering more if you need to scale your service quickly due to large demand spikes.#2019-05-0217:19kszaboIn the end I took your advice and did not AOT before uberjaring. Still I stayed with badigeon because it allowed me to exclude some cljs related files I don’t need from my resources, which sadly depstar doesn’t support, having a static exclude function.#2019-05-0217:24seancorfieldFeel free to create an issue for depstar to exclude files -- I just refactored some of that code recently so it'll be easier to extend to support options (although, right now, depstar has no command-line options at all beyond the JAR filename).#2019-05-0215:56avi👋 I just searched this channel, the Web, and the Clojure JIRA for anything related to git dependencies that use submodules, but didn’t turn up anything… I’m trying to include a dependency via git, and when I run clojure -Stree it fails with org.eclipse.jgit.api.errors.JGitInternalException: Checkout conflict with file: resources/devtools-protocol which is caused by org.eclipse.jgit.errors.MissingObjectException: Missing unknown 40a14030 (sha truncated) — that’s the sha of the submodule.
Anyway I’m not looking for free troubleshooting, just wondering whether this is a known issue and whether it’s on anyone’s radar? If not I could create a ticket…?#2019-05-0215:57dominicmI don't think this has come up 🙂 a jira is best I think#2019-05-0215:57avialrighty — thanks!#2019-05-0216:05avibug report: https://dev.clojure.org/jira/browse/TDEPS-126#2019-05-0320:01vlaaadI really want to have add-lib branch into master, dynamic dependency loading removes so much friction!#2019-05-0320:29dominicmIt doesn't need to be on master#2019-05-0320:41seancorfield@vlaaad you can depend on the SHA for it -- that's what I do. Alex generally keeps that branch up-to-date with master.#2019-05-0320:42seancorfieldhttps://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L106-L119#2019-05-0320:56vlaaadI know I can depend on shas, but that branch is 18 commits behind#2019-05-0320:58vlaaadoh, latest commit was merging master into it, nice#2019-05-0320:59vlaaadanyway, being unofficial and partially supported is concerning: I want newest stuff as well#2019-05-0321:19dominicmDevelopment workflows can be a little more alpha, especially something low risk like this. If it doesn't work, you restart the repl instead.#2019-05-0409:20yuhanhow do I add jvm-opts via the clj command?#2019-05-0409:21yuhanTrying to get https://github.com/clojure-goes-fast/clj-async-profiler working#2019-05-0409:21yuhanclj -Djdk.attach.allowAttachSelf does not work#2019-05-0409:28yuhanright, I figured it out - clj -J-Djdk.attach.allowAttachSelf#2019-05-0409:32yuhanbut to add it to my deps.edn file I can't just insert :jvm-opts ["-Djdk.attach.allowAttachSelf"] at the top level?#2019-05-0410:43alexmillerNo, that’s an outstanding issue. You will need to put it in an alias and use the alias#2019-05-0614:10tianshuI want use -Jopt to add a jar file into classpath. what is the correct syntax looks like? clojure -J-cpjarpath ... seems to be no effect and clojure -J-cp jarpath ... will cause an error.#2019-05-0614:22dominicm@doglooksgood use -Sdeps#2019-05-0614:27alexmillerclj builds the classpath so you must use your deps.edn for any modifications to the classpath. in this case, you'll want to use a local jar lib dependency - like my/lib {:local/root "/path/to.jar"}#2019-05-0614:28alexmillerif you just want to add it sometimes, either put it in an alias with :extra-deps or declare it dynamically on the command line with -Sdeps#2019-05-0618:43sparkofreasonGetting this error running clojureL com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available. This is on WSL running under Windows 10. Seem to recall having this issue before, but I don't remember the fix. git commands work fine, I recall this being something specific with the Java ssh implementation.#2019-05-0618:51sparkofreason@seancorfield This was encountered while attempting to build an uberjar with depstar. Tried on Windows instead of WSL, where I get Exception in thread "main" java.nio.file.NoSuchFileException: C:\Users\dave\AppData\Local\Temp\uberjar8947452084826452117\uncomplicate\neanderthal\aux.clj, which seems weird, like there was a file in the manifest that didn't get copied.#2019-05-0618:54seancorfield@dave.dixon If you can repro easily, feel free to create an issue on GitHub with details and I'll take a look -- but I suspect something AOT-related#2019-05-0622:30seancorfield@dave.dixon Thanks for the repro case. It's definitely a weird one. Files/copy is failing on the target file during the copy which suggests the parent folder is missing (it isn't) or there is a permissions problem (there isn't). I've tried various changes to depstar locally but cannot figure out what the issue is.#2019-05-0622:32seancorfieldTracing through the JDK source, it seems to hinge on the last Windows filesystem error triggering an exception so I thought maybe it was a sync'ing issue -- but using SYNC or DSYNC for StandardOpenOption made no difference either.#2019-05-0622:36sparkofreasonI cloned and ran with :local/root and it works. 🤔#2019-05-0622:40sparkofreasonWait - never mind, I was commenting out the neanderthal dependency 😬 Been that kind of day.#2019-05-0623:16sparkofreason@seancorfield If I exclude that file explicitly, it "works", in the sense of not throwing that exception. So it seems to be that single aux.clj file that's an issue.#2019-05-0623:17seancorfieldWeird.#2019-05-0623:46seancorfieldAnd it gets weirder. So I modified depstar so that if a copy operation fails, it tries again, just copying the file to the top of the tree in the temp folders (so uncomplicate/neanderthal/aux.clj copies to thing.clj) and then it "succeeds" and copies the files that follow aux.clj in the Neanderthal JAR. However, I got a weird NoSuchFile exception trying to write test.jar after that (where it shouldn't matter if the file exists or not). I ran touch test.jar and re-ran the command -- and it worked just fine (with aux.clj copied to the top of the tree).#2019-05-0700:17alexmillerare there two aux.clj files somehow?#2019-05-0700:17alexmillersuch that you're hitting a locked file or something?#2019-05-0700:18alexmillerjars can contain two files of the same name or a file and a directory of the same name#2019-05-0700:20seancorfieldNope. Just the one aux.clj#2019-05-0700:22seancorfieldIt's really odd because the exception (no such file) comes from a copy operation for the target file -- which doesn't exist because that's the file you're trying to copy to#2019-05-0700:23seancorfieldand it succeeds in copying additional files to that same folder location after failing on this one file.#2019-05-0700:24seancorfieldAnd you can copy the file to some other location in the tree...#2019-05-0700:24alexmillerwell, file systems are a new fangled technology, wouldn't expect them to be foolproof yet#2019-05-0700:24alexmillerif only they were statically typed, we wouldn't have these bugs#2019-05-0700:29seancorfielduncomplicate/neanderthal/internal/device/common.clj
uncomplicate/neanderthal/internal/device/cublas.clj
uncomplicate/neanderthal/internal/device/clblock.clj
uncomplicate/neanderthal/aux.clj
java.nio.file.NoSuchFileException C:\Users\sean\AppData\Local\Temp\uberjar11308132828377820906\uncomplicate\neanderthal\aux.clj
[u n c o m p l i c a t e / n e a n d e r t h a l / a u x . c l j]
[117 110 99 111 109 112 108 105 99 97 116 101 47 110 101 97 110 100 101 114 116 104 97 108 47 97 117 120 46 99 108 106]
java.nio.file.NoSuchFileException C:\Users\sean\AppData\Local\Temp\uberjar11308132828377820906\_random.txt -> C:\Users\sean\AppData\Local\Temp\uberjar11308132828377820906\uncomplicate\neanderthal\aux.clj
failed to copy back over
uncomplicate/neanderthal/block.clj
uncomplicate/neanderthal/linalg.clj
uncomplicate/neanderthal/real.clj
uncomplicate/neanderthal/core.clj
uncomplicate/neanderthal/opencl.clj
uncomplicate/neanderthal/internal/device/vect-math.cl
uncomplicate/neanderthal/internal/device/blas-plus.cl
It successfully copies files before and after. It can copy that one file to _random.txt happily but still can't copy it to the uncomplicate\neanderthal\aux.clj location after that.#2019-05-0700:44seancorfieldI can even copy it to uncomplicate/neanderthal/not.aux.clj ... is a filename of aux.* somehow magical on Windows?#2019-05-0700:53seancorfieldYes! aux, con, and prn are special filenames on Windows! https://superuser.com/questions/206423/windows-7-can-not-rename-a-file-to-aux-svg-the-specified-device-name-is-inva#2019-05-0700:54seancorfieldSo I bet that is what is happening here. Wow...#2019-05-0700:55seancorfieldPS C:\Users\sean\clojure\depstar-test> touch aux.clj
C:\Program Files (x86)\Gow\bin\touch.exe: cannot touch `aux.clj': No such file or directory
PS C:\Users\sean\clojure\depstar-test> touch con.clj
C:\Program Files (x86)\Gow\bin\touch.exe: setting times of `con.clj': Invalid argument
PS C:\Users\sean\clojure\depstar-test> touch prn.clj
C:\Program Files (x86)\Gow\bin\touch.exe: cannot touch `prn.clj': No such file or directory
PS C:\Users\sean\clojure\depstar-test> touch prn1.clj
PS C:\Users\sean\clojure\depstar-test>
:rolling_on_the_floor_laughing:#2019-05-0707:25tianshu@dominicm @alexmiller thanks!#2019-05-0707:25tianshuvery clear#2019-05-0713:04sparkofreason@seancorfield @alexmiller Since I seem to be completely blocked on building on windows, any idea on my WSL issue above?#2019-05-0713:39mhuebertif one specifies a tools.deps version i deps.edn, is that version somehow used to resolve other deps?#2019-05-0713:41alexmillerno#2019-05-0713:42alexmillerthere isn't any way "official" way to change the tools.deps version you're using#2019-05-0713:43alexmillerthat said, you can edit your clojure script and modify the tools_cp to point to a different source version of tools.deps (and that's how I test a lot of stuff)#2019-05-0713:44alexmillerso instead of tools_cp="$install_dir/libexec/clojure-tools-1.10.0.442.jar", something like
tools_cp="/path/to/my/tools.deps.alpha/src/main/clojure:$install_dir/libexec/clojure-tools-1.10.0.442.jar"#2019-05-0713:45alexmillereventually I'll provided an env var for that so you can just set the var rather than hack the script#2019-05-0713:50mhueberthmm. i’m configuring a circleci build which has some git deps that need the recent local/root changes, currently it installs clojure via the Manifest type not detected when finding deps for chia/reactive in coordinate #:local{:root "/home/circleci/chia/reactive"}
& there is nothing in the /home/circleci/ dir, except form the main “/project” working directory#2019-05-0714:06alexmilleryeah, so it can't detect what kind of project that is as there is no manifest file (deps.edn or pom.xml)#2019-05-0714:07mhuebertso chia is in /home/circleci/.gitlibs#2019-05-0714:08mhuebertthe clj -Spath command is executed in the ~/project/editor dir#2019-05-0714:08mhuebertwhich has a deps.edn pointing to the :git dep#2019-05-0714:09alexmillerboth local deps and git deps try to detect their project type by looking for well known files at the project root#2019-05-0714:09mhuebertthe git dep is downloaded#2019-05-0714:10mhuebertit appears to be finding the root deps.edn at ~/.gitlibs/libs/chia/chia/da6ceb2989b712ab59c654679e1eb7b2e4c6ef02#2019-05-0714:11mhuebertor one of the sub-dirs.. have to check#2019-05-0714:11alexmillerthe error you have above indicates something is referencing a local dep at /home/circleci/chia/reactive#2019-05-0714:12alexmillerthat may be the resolved path of a relative dir in the referring deps.edn#2019-05-0714:13mhuebertthe root dep has chia/reactive {:local/root "reactive"}, and some sibling deps have chia/reactive {:local/root "../reactive"}#2019-05-0714:13mhueberti have no idea how it is navigating so far up#2019-05-0714:14alexmillerits based on the working directory#2019-05-0714:16mhueberthm i thought that with the recent changes, one could use relative paths within a git dep, and those would be resolved not from the current working directory, but relative to the path into the git dep where the relevant deps.edn is being read?#2019-05-0714:21alexmilleryes, should work that way. I don't have enough context to understand where things are going awry for you#2019-05-0714:21mhuebertok. i will keep digging, thank you for the pointers#2019-05-0714:22alexmillerclj -Stree might help, not sure#2019-05-0717:10dwhite44i have a project that is using a git reference to a private repo. everything works fine locally (mac) but when I try anything on our build server (CentOs), I get “Error building classpath. connector is not available:
com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available”. this happens just by running /usr/local/bin/clojure. If I comment out the dependency to that git repo, it works fine and brings in all the maven dependencies. I can clone that repo without issue so doesn’t seem like it would be a git access issue. any ideas?#2019-05-0717:57alexmillerit's trying to use the ssh agent, which does not seem to be running#2019-05-0717:59tanzoniteblack@dwhite44 a number of people in our company have had this issue since we switched to tools-deps. If you're on a mac, make sure you've explicitly run ssh-add -K in the terminal. If that still doesn't work, try removing your ssh config (i.e. mv ~/.ssh/config ~/.ssh/config.bak). Some of the people in our company have had something in their config that jgit chokes on. Never have figured out exactly what#2019-05-0718:00dwhite44ok. just not sure how the same repo would clone fine if the ssh agent isn’t running#2019-05-0718:02alexmillerthere are several possible reasons due to the differences between how git finds credentials vs how jgit+jsch does it#2019-05-0718:02tanzoniteblack^^^ That has been a major source of headaches#2019-05-0718:04sparkofreasonI have ssh-agent running, added the key, no config file, same error.#2019-05-0718:13alexmilleryeah, Ghadi Shayban has done a full rewrite of tools.gitlibs that just shells out to git, which has different tradeoffs but at least generally "does what git does"#2019-05-0719:59seancorfieldI just pushed depstar "0.2.0" to clojars -- it's a rewrite to leverage the ZipFileSystem, as a way to get around Windows' reserved filename issues, which means a single copy pass so it's also much faster. Would love feedback from anyone already using depstar.#2019-05-0720:00seancorfield(although it now produces slightly different ownership/permissions on the resulting JAR file, I think because of the FileSystemProvider approach)#2019-05-0910:18kszabohey, how do you manage compilation of Clojure :gen-class files with tools.deps at development time? Do you call clojure.core/compile manually?#2019-05-0916:52dominicmI do, yeah#2019-05-0917:53seancorfield{:deps {org.clojure/clojure {:mvn/version "1.10.0"}}
:paths ["classes" "src"]
:aliases
{:compile {:main-opts ["-e" "(compile,'cfml.struct)"]}
:build {:extra-deps {seancorfield/depstar {:mvn/version "RELEASE"}}
:main-opts ["-e" "(compile,'cfml.struct)"
"-m" "hf.depstar.jar" "cfml-interop.jar"]}}}
is an example from one of my projects. I run it with clj -A:compile to get a REPL or clj -A:build to create a JAR for deployment. It duplicates the compile call in both. I could just have it in :compile but then I'd have to remember clj -A:compile:build whenever I wanted to build a JAR.#2019-05-1009:48kszaboThanks! We have a bit more complicated project with shadow-cljs + deps.edn with docker images. Since we already had a Makefile to run docker-based integration tests locally, we just added a new target to it, essentially doing what you are doing above with a manual clojure invocation.#2019-05-0914:51hlolliin leiningen there's a possibility of defining native paths with :native-path, that causes all files on this path to be spit'ed out into the current working directory, which suites well for jna/jni apps using .so files, I'm not seeing this behaviour in tools.deps, is it missing, not implemented or am I missing an argument?#2019-05-0915:44kszaboThis is implemented in badigeon:
https://github.com/EwenG/badigeon/blob/master/sample/badigeon/sample.clj#L121#2019-05-1015:02hlollivery nice, thanks!#2019-05-0914:59alexmillerNot implemented#2019-05-0915:02hlolliprima 🙂 no stress, then I'll continue to advise to Overtone users to stick with leiningen until then#2019-05-0918:32borkdudeis this kind of thing possible with tools.deps? https://stackoverflow.com/a/24949419/6264
I imagine it’s just a -e expression?#2019-05-0918:41seancorfield(! 1084)-> clj -e '(set! *assert* false)' -r
false
user=> (assert false)
nil
user=> (assert true)
nil
user=> ^D
Thu May 09 11:40:55
(sean)-(jobs:0)-(~/clojure)
(! 1085)-> clj -e '(set! *assert* true)' -r
true
user=> (assert false)
Execution error (AssertionError) at user/eval138 (REPL:1).
Assert failed: false
user=> (assert true)
nil
user=>
#2019-05-0918:41seancorfieldWhy "imagine" when you can just "repl"? 🙂#2019-05-0918:43borkdudetrue. I’m just wondering if there was a java property for it#2019-05-0918:43borkdudeI’ll just put an assert in my code and see if it fails, thanks#2019-05-0918:43seancorfieldThere's a JVM property for spec asserts.#2019-05-0918:44seancorfield(but not regular Clojure asserts I believe)#2019-05-0918:44borkdudeyeah, that’s why I imaged there would be one for regular asserts as well#2019-05-0918:48borkdudeworks 🙂#2019-05-0919:14borkdude:jvm-opts are only supported in aliases, not on the top level right? I don’t see this in the documentation and I remember asking this before#2019-05-0919:14ghadiright ^#2019-05-1006:59mpenetanother uberjar lib I discovered today : https://github.com/tonsky/uberdeps#2019-05-1010:22rickmoynihanCool. I’ve added it to the Tools wiki page here: https://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging#2019-05-1021:33timgilbertSlightly off-topic, but this new GitHub package repo thing looks potentially interesting for tools.deps enthusiasts, it supports Maven packages: https://help.github.com/en/articles/configuring-maven-for-use-with-github-package-registry#2019-05-1110:40luposlipHey @alexmiller - any chance this will get fixed soon?
https://dev.clojure.org/jira/browse/TDEPS-46#2019-05-1110:40luposlipAnother example of the issue:
https://github.com/luposlip/json-schema/issues/1#2019-05-1110:51olivergeorgeAre there any tips for debugging TransportException "Auth fail" errors? I swear I haven't changed anything and I'm suddenly getting this error. I can manually git clone the repo via ssh just fine.
Error building classpath. : Auth fail
org.eclipse.jgit.api.errors.TransportException: : Auth fail
Full stack trace here: https://gist.github.com/olivergeorge/17105466d16021269415ff66423a91ab#2019-05-1314:14borkdudeI’m running a project which calls the tools.deps API on Windows, but this crashes (clojure on Windows installed in PowerShell per wiki):
https://www.dropbox.com/s/1ouyqublibscp3x/Screenshot%202019-05-13%2016.12.23.png?dl=0
https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/reader.clj#L21#2019-05-1314:20borkdudeMore info here: https://github.com/taylorwood/clj.native-image/issues/6#2019-05-1314:37alexmillercan you file a jira on TDEPS?#2019-05-1314:51borkdudehttps://dev.clojure.org/jira/browse/TDEPS-128#2019-05-1315:50timgilbertYou might want to take a look at https://dev.clojure.org/jira/browse/TDEPS-120 @borkdude, I was seeing the same error message when my classpath was too long#2019-05-1315:52timgilbertHmm, actually looking more closely I think your issue is something different, never mind#2019-05-1413:47stathissideriswhen using a git dependency, do I also get access to the resources of the dependency or just the source?#2019-05-1414:40dominicm@stathissideris what's the distinction?#2019-05-1414:40dominicmIf its in :paths, you get it#2019-05-1414:41stathissideris@dominicm yeah, I just figured that out by experimenting 🙂 thanks!#2019-05-1514:32ChrisIs anyone aware of a tools-deps dependency visualizer? (i.e. the equivalent of https://github.com/walmartlabs/vizdeps)#2019-05-1516:32ChrisNevermind! It turned out to be relatively easy to write my own 🙂#2019-05-1517:54richiardiandreaA little rusty with Clojure on JVM...if I want to work against a Java code base, do I need to include taarget in order to import classes in the REPL using clojure?#2019-05-1517:55alexmilleryes, assuming target is the root of your compiled classes#2019-05-1517:56richiardiandreaok cool that's what I was missing then, I guess it goes without saying that I then need to compile the java code every time the file changes#2019-05-1517:59richiardiandreathank you Alex that works#2019-05-1520:18holyjakHello, what do people use to watch for file changes to trigger actions (e.g. clojure.core/compile)? Lein certainly has a watch functionality/plugin but what about clj CLI? Thanks!#2019-05-1520:20alexmilleras above, clj doesn't cover compilation#2019-05-1520:20alexmillerit does cover "running programs"#2019-05-1520:20alexmillerthe compiler is a program#2019-05-1520:20alexmillerwatching is a program#2019-05-1520:21alexmillercompose programs, run with clj if you like#2019-05-1520:38holyjakthank you, Alex. My question is essentially: Does anyone made a clj-friendly plugin for watching files and executing actions upon change?#2019-05-1813:42orestisI like hawk for this. #2019-05-1520:40alexmillerI'm not aware of something specific to tools.deps, but there several watcher type things out there#2019-05-1520:40alexmillerlast time I needed one, I just wrote the 60 lines of java interop to use the one in the jdk#2019-05-1520:41alexmilleradmittedly, it's a weird java api#2019-05-1520:44alexmillerhttps://www.clojure-toolbox.com/ has a bunch of filesystem watcher libs#2019-05-1520:53souenzzo@holyjak I use (load "core") to load/reload files
But usually it's a "built-in" feature from your IDE/Plugin/Editor/..#2019-05-1521:07holyjakwell, I need to watch for changes to run compile automatically...#2019-05-1521:08seancorfieldPersonally, I don't like watcher-based workflows. I usually find the watch fires when I don't want it to and/or gets into a bad state because it fires while there are still errors in the code.#2019-05-1521:09seancorfieldI'd much rather have an explicit compile command in a (comment ...) form that I can eval from my editor without even worrying about what the terminal is doing.#2019-05-1521:12seancorfield(also, why do you need to run compile at all? what features are you relying on that require Clojure code to be compiled?)#2019-05-1522:23holyjakhttps://github.com/holyjak/clj-concordion - integration with Java Specification by Example test framework that expects me to give it Fixture objects with test-specific methods. So you do `(deffixture "class.Name" [fns2expose as methods]) for each specification document.#2019-05-1522:24holyjakgood points about the downsides of watchers, thank you!#2019-05-1522:25seancorfieldAh... so you're relying on code gen / gen-class rather than reify for this?#2019-05-1522:26holyjakexactly ,as there is no interface to reify#2019-05-1521:10holyjakWied problem: while this works clj -J-javaagent:bin/springloaded-1.2.5.RELEASE.jar -J-noverify
this fails: ; deps.edn: [...]
:aliases {:agent {:jvm-opts ["-J-javaagent:bin/springloaded-1.2.5.RELEASE.jar" "-J-noverify"]}}
; CLI:
$ clj -O:agent
Unrecognized option: -J-javaagent:bin/springloaded-1.2.5.RELEASE.jar Why?#2019-05-1521:11seancorfield-J passes options through. They don't start with -J inside :jvm-opts#2019-05-1521:11seancorfieldYou want :aliases {:agent {:jvm-opts ["-javaagent:bin/springloaded-1.2.5.RELEASE.jar" "-noverify"]}}#2019-05-1521:12holyjakoh I see, please ignore 🙂 I have to omit -J#2019-05-1521:20holyjakGiven a .clj file path, what is the simples way of finding out the namespace of the file? I could read the first line and check for #"\(ns (\w+)" but there is surely a cleaner way?#2019-05-1521:24seancorfieldThere's no guaranteed way because you can change namespaces inside a file and you can load arbitrary source files into another namespace.#2019-05-1521:26seancorfieldThe convention is path/to_the/file.clj => path.to-the.file tho' -- note the _ to - mapping. And you'll need to know what it is _relative to your classpath_ because if src/main is on your classpath, src/main/path/to_the/file.clj is what will get loaded if you try to require path.to-the.file#2019-05-1521:32holyjakthank you!#2019-05-1521:40alexmillertools.namespace is a lib for doing this kind of thing#2019-05-1718:23sogaiui have an alias for starting a socket repl in my ~/.clojure/deps.edn -- does anyone have a good way to get the port chosen to be somewhat random and then print that port number to stdout so i know where to connect to?#2019-05-1719:00lilactownI don’t think you can parameterize over aliases like that, but you could write a small script around it perhaps?#2019-05-1720:00alexmillerrather than relying on the startup params, you could just write a small .clj that programmatically chose a random port and started the server#2019-05-1720:00alexmillerand then have the alias run that#2019-05-1720:01alexmillerthis is really just a call to https://clojure.github.io/clojure/clojure.core-api.html#clojure.core.server/start-server#2019-05-1720:01alexmillerwith the same params you pass on the cmdline mostly#2019-05-1720:56sogaiuthanks for the ideas, will consider#2019-05-1722:07seancorfieldAs an example, here's what we do at work.#2019-05-1800:12sogaiuthanks for the sample code!#2019-05-1722:08seancorfieldThat also starts Cognitect's REBL if it's on the classpath. And it saves the Socket REPL port in a dot file for next time.#2019-05-1722:08seancorfield(to sort of match what nREPL tools do)#2019-05-1812:27holyjakIs there any trick for using deps.edn with a :local/root dependency to make something like "~/.m2/repository/.../some.jar" work without having to hardcode the actual path to the $HOME directory? The beauty of ~/ is that it is quite cross-platform...#2019-05-1813:15alexmillerIf it’s in your m2 repo, then why not use a Maven coord?#2019-05-2017:33Aaron CummingsIs there a flag for clj to force a refresh from the Maven repos, similar to the -U flag on lein?#2019-05-2017:50alexmillercurrently, no#2019-05-2017:50alexmillerthere is an outstanding ticket for this#2019-05-2017:51alexmillerone option is to wipe the portion of your ~/.m2/repository as that's where the cache of update times is held#2019-05-2017:52alexmillerI'd recommend being as selective as possible in deleting parts of ~/.m2/repository#2019-05-2017:55Aaron CummingsI did work around this with the 'mv /.m2 /.m2.to-be-removed' nuclear option. I'll try to find that ticket and upvote it.#2019-05-2018:03Aaron CummingsFound it - TDEPS-62#2019-05-2020:05Aaron Cummings(not sure if it makes sense to upvote in the old Jira)#2019-05-2020:12alexmillerwell it won't hurt, but also it won't help :)#2019-05-2020:13alexmillerit's ok, I am aware of the issue and have a plan for it :)#2019-05-2021:00Aaron CummingsThanks Alex.#2019-05-2116:25didibusWhen using a git repo, can you make it depend on a particulars branch head instead of a specific sha?#2019-05-2116:25didibusSay I always want the latest commit#2019-05-2116:26didibusMy idea would be to have a release branch, where each commit is a new stable release#2019-05-2116:26seancorfield@didibus Take a look in my dot-clojure file -- I give an example of how to resolve master HEAD to a SHA in there.#2019-05-2116:27seancorfieldBasically, tools.gitlibs provides tag/branch resolution#2019-05-2116:29didibusHum#2019-05-2116:30didibusI see, but that's basically the same as the user going to github to lookup the latest sha, just instead he can find it from inside a Clojure repl#2019-05-2116:30didibusOr asking him to always dynamically inject the dependency#2019-05-2116:31seancorfieldYou can hide it away behind a bit of code, but yes, it requires a lookup and a dependency load (my example is via add-lib).#2019-05-2116:31didibusI was hoping for something where I can just say, add this alias and done. Now the user always gets the latest release, so my CLI is pretty much auto-updating.#2019-05-2116:32didibusLike how mvn/version can be "RELEASE"#2019-05-2116:32seancorfieldYeah, it would be nice if :sha (or an equivalent) could specify a label to resolve first -- but that goes against repeatability.#2019-05-2116:33seancorfieldAnd I believe even Maven's RELEASE is considered deprecated? Or at least somewhat bad practice...#2019-05-2116:35ghaditags are references, tools.deps only deals in values#2019-05-2116:35alexmillerusing a changing version destroys the ability to cache and select versions#2019-05-2116:35alexmillerso it's intentionally not supported#2019-05-2116:36didibusYes, and I understand it in the Java world. I think Release is deprecated or even removed in Maven 3.x. But in the Clojure world, tools.dels is also used as a minimal package manager for Clojure build tools and CLI apps. For that, repeatability isn't a concern, but auto-update is.#2019-05-2116:37didibusYour dot-file demonstrate that use case.#2019-05-2116:38alexmillerauto-update is not a use case tools.deps is trying to cover right now#2019-05-2116:38didibusHum... Can't you still cache the sha of the head? Or you mean, the cache won't be as good since it will always need to check the repo for the latest commit sha?#2019-05-2116:38alexmillersha's don't change meaning#2019-05-2116:40didibusThe workaround I'm thinking is to wrap my CLI in another Clojure dependency that performs a dynamic add-lib and resolves latest. A kind of get-latest shim. But I kinda hate it#2019-05-2116:41didibusThat one would never need updating (hopefully), and would always just bootstrap my latest#2019-05-2116:42alexmillernot saying this isn't a use case worth considering, just not one we are trying to cover right now#2019-05-2116:42didibusGot it#2019-05-2116:43didibusI can live without it and patiently wait. Just thought I'd see if there was already something of that sort.#2019-05-2116:44didibusI might be able to create an update command on the cli itself that edits the deps.edn file to its latest sha. That might be cleaner.#2019-05-2116:49didibusI am very pro for reproducible builds. And now that I'm thinking about it, I can't see good reasons for local deps.edn to not be version specific, even in their version of build tooling, test runner, etc. Since you want everything exactly the same to reproduce. But for the global deps.edn, which is more for installing Clojure utils on your machine, which as I see it, is like a package manager for Clojure apps, such a feature would be great.#2019-05-2116:50alexmillerYeah#2019-05-2116:51alexmillerAlthough you’d be pretty unhappy if your tool auto updated and broke stuff#2019-05-2116:53lilactownyeah I like the fact that I can pin the tools to a specific sha. makes troubleshooting much easier#2019-05-2117:03seancorfieldI only use RELEASE for dev tools locally, although I made it the default in new projects created with clj-new -- on the assumption that when you start off, getting the latest versions is fine but you'd change it to a specific version as soon as you start to do any serious work.#2019-05-2117:04seancorfield(and I didn't want to get into the situation of lein where it often lags way behind Clojure's releases and you need a whole new lein release just to bump the Clojure version in new projects)#2019-05-2117:08lilactowngood point seancorfield#2019-05-2117:21didibusHum... Ya. I'm wondering if a tools.deps tool could itself suffice. Like just have a -A:update-latest-deps#2019-05-2117:22didibusWhich would resolve latest maven and git releases and update your tools.deps file.#2019-05-2117:22didibusSo the user is in charge of when to update#2019-05-2117:22didibusIt could even have a rollback feature#2019-05-2117:22didibusAnd what not, to make it more resilient in the case of breakage#2019-05-2117:24didibusAlso, another thought that crossed my mind, and I'm just brainstorming at this point. I've noticed my default usage of Clojure cli is -A. Would it make sense then to default all unknown commands to be treated as an alias?#2019-05-2117:24didibusSo clj new bla would be treated as clj -Anew bla#2019-05-2117:25didibusLike when it's not a valid path to a script#2019-05-2117:26seancorfieldMost likely you would not want all Git deps updated, just certain ones.#2019-05-2117:27didibusYa, I'm sure it could be enhanced with some options to exclude or include. Or even made interactive so it asks you Y/n for each one you want to update#2019-05-2117:26didibusOr to be honest I'd rather have to use say -s for running scripts, and aliases be the no option default#2019-05-2117:27seancorfieldIt's kind of important to have the -R/`-C`/`-M`/`-O` alias options tho' right? -A is convenient but kinda lazy.#2019-05-2117:29didibusHum.. Ya I was wondering that. I guess I'm very much thinking here about the specific use case where tools.deps is used as an app launcher for Clojure and a build script launcher.#2019-05-2117:30lilactownI won't speak for alexmiller / et. al. but it's my impression is that they are only interested in solving "resolving dependencies and constructing the classpath" right now#2019-05-2117:32didibusYa, I'm kinda just wishy washy throwing some ideas which I haven't put much thought into. So none of this is criticism or even feature request, since I'm not even convinced any of this is a good idea yet.#2019-05-2117:33didibusIt's just that, the more I think about "resolving deps and constructing the classpath" the more I realize that's basically all you need for being a package manager and app launcher for Clojure.#2019-05-2117:36jmvhi all, i'm trying to get a coworker setup with a clojure project and we are hitting this error
org.eclipse.jgit.api.errors.TransportException:
is this an ssh auth error?#2019-05-2117:52jmvto follow up, we found the culprit. it was a bad entry in the known hosts file.#2019-05-2117:38didibusThe only real part missing is the idea of a "package". Its technically what an alias is. It's a preconstructed set of dependencies and main entry point which together consists of a working release of an app. But currently, aliases aren't easily shareable. Like you can't publish them, or update them easily. And launching them is a little ackward due to the use of -A.#2019-05-2117:42seancorfield@didibus The one thing I really "miss" with aliases is the abilty to compose them in deps.edn. Otherwise you either need to duplicate stuff under multiple aliases or the user has to specify multiple aliases on the command line.#2019-05-2117:43seancorfield(and I do generally use -A out of convenience rather than caring about whether I need -R or -M -- even tho' for things like test-runner, being able to use -R:test vs -R:test -M:test or -A:test would be "better practice" in a way)#2019-05-2117:44didibusHum.. I feel maybe it's that aliases are currently used as packages, where really they were meant as ways to optionally add more or less dependencies to your running code, for development purposes mostly.#2019-05-2117:44didibusMaybe adding something like -P would make more sense for these use cases.#2019-05-2117:45didibusWhere its clear you won't be changing the set of dependencies, but fully using the one defined by it#2019-05-2117:45seancorfieldI don't see the "alias as package" similarity at all... puzzled#2019-05-2117:46didibusSo, I might want to inject a test runner and also have my src on the path and run that. So I create an alias to inject the test runner. That's one use case.#2019-05-2117:48didibusBut maybe I want to be able to run a command line editor implemented in Clojure. So I add an alias to it in my global deps.edn file and then to run it I do clojure -Acljedit bla.txt#2019-05-2117:49didibusThis command installs the cljeditor app on my machine, and launches it. On the next run, the install step is skipped (because of caching).#2019-05-2117:49didibusIn this case, I'm not interested in composing classpath dependencies. I'm not even coding. I want to install and run a Clojure app.#2019-05-2117:50didibusIn that way, the alias is defining a package in the sense of Clojure app.#2019-05-2117:53didibusWhen tools.deps is used as a replacement for lein or as a build tool. Its kinds doing the same thing as well. In that you're adding all these convenient Clojure cli tools to your development environment. The difference is some of them also leverage tools.deps as the way to add your own code sources and resources which the build tool needs on the classpath as well.#2019-05-2117:54didibusWhich is where composing classpath comes into play I guess.#2019-05-2117:56dominicmIt would be really cool to have an upgrade bisect where an attempt is made to update your dependencies and then run the tests. If they still pass you're good, otherwise you remove the upgraded dependency which caused the tests to fail and try again#2019-05-2117:56didibusThat's a great idea#2019-05-2117:57alexmillerThis does not need to be in tools.deps#2019-05-2117:59didibusYa, most likely is best as a tools for it. Could be added to one of the existing test-runners#2019-05-2118:01dominicm@didibus I was actually thinking that a tool which processed your deps.edn should take a "test" script to run that as long as it doesn't throw (or match whatever the clojure.test failure format is)#2019-05-2118:02didibusThat's even better, that way its decoupled from the concrete test-runner and might be extended to work for all of them.#2019-05-2118:24dominicmOr even an arbitrary piece of code#2019-05-2118:25dominicm"This code used to return 10, which dependency changed that?"#2019-05-2118:26alexmillerwould intersect well with some stuff Rich (in the form of codeq and REBL) and me (in the unreleased spec-aware testing tool) have worked on with respect to fine-grained code dependencies#2019-05-2118:30seancorfieldears perk up... "spec-aware testing tool"? Mmm... sounds lovely!#2019-05-2118:30alexmillerI've talked with you in the past about it, I'm pretty sure#2019-05-2118:30seancorfieldIt's been a while tho'... glad to hear work is still ongoing...#2019-05-2118:31alexmillerit's not. I mean it's not dead, but I haven't touched it in 2.5 yrs#2019-05-2118:32seancorfieldhaz a sad Hopefully it will eventually see the light of day?#2019-05-2118:32alexmillerbut it was always part of the original spec idea and may eventually get revived#2019-05-2118:32alexmillerit may make more sense to build on top of codeq 2 whenever I get back around to it (which is also not released, but is in active development)#2019-05-2118:33alexmillerand starting to get real interesting when combined with some new things coming in REBL#2019-05-2118:34alexmillerand Datomic, and Clojure itself, now that I think of it#2019-05-2118:34seancorfieldI'm always happy to see new features in REBL since it's a core part of my day-to-day workflow now.#2019-05-2118:34alexmillernext release will have some great new stuff in it, Rich has been in the hammock :)#2019-05-2117:57didibusAnyways, I'll think about all this some more and I'll stop with my thought dumping :drooling_face:#2019-05-2118:25alexmillerone of the goals with introducing clj was to encourage more "small" build programs. I think that's been pretty successful. I think it's also useful to step back with where we are now and look at how people are using those, what use cases/problems we could support better, and what we could do to help. Eventually, we'll get around to that.#2019-05-2119:19dominicmRunning multiple programs seems to be the difficulty people keep stumbling over, from my observations at least.#2019-05-2119:21alexmillerlike?#2019-05-2119:22alexmillerlike a static set of things, or something where there is same input to all of them?#2019-05-2119:27dominicmFigwheel, and then nrepl are the two I mostly see. And then combining them with rebel.
It's not well suited to multiple mains because they need to background (except for rebel). But I also consider most of them individual choices rather than project ones.#2019-05-2119:31seancorfieldThat's why we wrote our build script: it takes a series of arguments and runs clojure for each one (plus some alias manipulation).#2019-05-2119:32seancorfieldBut the multi-main in one JVM is another use case that could be nice as a built in. It's no big deal to write your own -main to start up multiple things tho'...#2019-05-2119:33seancorfieldWe have a dev.repl/-main that starts up various combinations of tooling for that.#2019-05-2119:45alexmillerwhat if clojure.main took multiple -m's ?#2019-05-2119:51dominicm@alexmiller it might work, I've done a little thinking, and I'm guessing that most tools would need to take a -d flag.
The other side of this is "activation", the interface needs to be supportive to a user combining their own development programs with those of the project.#2019-05-2119:51alexmillerwhat's -d?#2019-05-2119:52dominicmDemonize. Sorry.#2019-05-2119:52alexmillerby "multiple -m"'s, I assuming each would have their own main + args#2019-05-2119:52alexmiller-m foo 1 2 -m bar 3 4#2019-05-2119:53alexmillerso if foo or bar took -d, that would be fine#2019-05-2119:53dominicmMe too. I just mean that eg nrepl would need a flag to indicate it should background itself.#2019-05-2119:53alexmillerah, unless they launched in parallel in different threads :)#2019-05-2119:54alexmillerdefinitely things to work through, but seems non-crazy to me#2019-05-2120:13seancorfieldIt would definitely be interesting if multiple -m options launched the first N-1 in threads and the last one ran directly -- and then it waited on all those threads to complete.#2019-05-2120:18ghadino thanks#2019-05-2120:19ghadisounds like that can be done in userspace#2019-05-2120:19ghadiif there were multiple mains, I'd expect the -main functions to return before moving on to the next one#2019-05-2120:27seancorfieldBoth options have their pros and cons. Maybe a -d option to run -main in a thread and -m to run -main sequentially is a good approach?#2019-05-2120:20ghadithere are so many choices in this neighborhood#2019-05-2120:20alexmillerI wouldn't leap to that conclusion#2019-05-2120:21ghadithe conclusion i deleted? simple_smile#2019-05-2120:22WhittlesJrSorry to interrupt... is there a feature similar to lein clean in Clojure CLI?#2019-05-2120:25seancorfieldclj -Sforce is probably the closest.#2019-05-2120:26seancorfieldIt ignores the cache, recalculates the classpath etc, and updates the cached values from that.#2019-05-2120:26WhittlesJrHmm, ok. Thanks!#2019-05-2120:28seancorfieldAlthough I guess you can get -d already with -e "(future ((requiring-resolve 'some.ns/-main)))" ... except for argument processing#2019-05-2120:37hiredmanof course then you have to decide who gets to call shutdown-agents#2019-05-2120:39dominicmI guess the thread needs to have interruption configured for shutdown#2019-05-2120:39dominicmOr basically daemonization#2019-05-2120:41alexmillerlet's just reinvent pods! :)#2019-05-2120:41seancorfieldOr not... 🙂#2019-05-2120:45dominicmBoot is coming for clj I believe, so maybe that will happen.#2019-05-2200:47hlollimaybe bit late to realize, that using tools.deps to resolve deps with any other build-system, requires clojure tools to be installed. I bumped into this https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/reader.clj#L21 where I got an error on a windows machine. But I wonder, is this really necessary, can't every function from the clojure binary be called from another clojure code?#2019-05-2200:49seancorfield@hlolli That's the easiest (and only supported) way for tooling to get the list of system, user, and local deps.edn files.#2019-05-2200:49seancorfieldOn Windows, you'll either need to use the alpha PowerShell installer for the Clojure tools, or stick with WSL and one of the "bash on Windows" Linux options.#2019-05-2200:50hlolliyes, easy to fix, but this makes releasing libraries with lein+tools.deps a no-go#2019-05-2200:50seancorfieldI'd recommend not using tools.deps with other build systems (as the author of boot-tools-deps) and just go with clojure / clj directly instead.#2019-05-2200:50hlolliI wonder if the function for describe wasn't originally written in clojure?#2019-05-2200:51hlolliyes I guess so, like I said, late to realize that limitation 🙂#2019-05-2200:51seancorfieldIn a very early version of tools.deps it did not shell out to clojure. It was changed to do that to get around a lot of other problems people were having with tooling.#2019-05-2200:53hlolliwell, tools.deps is still alpha, so let's see what happens. I'd root for keeping it as a library, no matter how much sweat and blood is needed (none of it coming from me 🙂 ). Then it also enables calling it straight from java without any deps.#2019-05-2200:56seancorfieldAt this point, it's pretty fixed in stone that it's going to shell out to clojure I think.#2019-05-2200:56seancorfieldThat's why there's been so much work done on getting a clojure command working on Windows (via PowerShell).#2019-05-2200:58seancorfieldI initially wrote boot-tools-deps thinking that combining Boot with clj/`deps.edn` was a good approach but once I started using it, I realized it really wasn't. Boot's ideas of paths and dependencies don't mesh well with the Clojure tools -- and it was just easier to stop using Boot altogether. I would imagine the same is really true of Leiningen as well.#2019-05-2200:59hlolliyes, I'm starting to think that. Or I start writing tool dependent code if lein x elseif tools.deps y. Which I've never done with clojure.#2019-05-2201:00seancorfieldThese days I'm just targeting clj/`deps.edn` but I'm also releasing libs to Clojars so they can be used by lein/`boot`.#2019-05-2201:01seancorfieldWe switched our entire stack from Boot to clj/`deps.edn` last year. We've been very happy with that.#2019-05-2201:01hlolliyes I bet, it's just so unneeded I think, to tell everybody to change build tool, even tough tools.deps rocks and I would want everybody to use it#2019-05-2201:03seancorfieldIf folks are happy with lein/`boot`, stay with lein/`boot`. Don't try to blend them tho' 🙂#2019-05-2201:11hlolliyes, I agree with the fact that the my world would be a better place if everybody used tools.deps#2019-05-2201:12alexmiller-Sdescribe doesn't need to shell out if there isn't an install deps.edn, which may happen#2019-05-2201:12hlolliah that explains! the mix here of project.clj and deps.edn in a directory is probably confusing#2019-05-2201:33seancorfield@alexmiller Tooling that calls clojure-env depends on tools.deps shelling out -- to run clojure -Sdescribe.#2019-05-2201:33seancorfieldAre you saying that tooling could figure out it doesn't need to call clojure-env?#2019-05-2201:35seancorfield@hlolli Which project are you looking at that has both project.clj and deps.edn? That usually means it's a Leiningen project that has added deps.edn so folks can rely on that project on GitHub directly from their own deps.edn...#2019-05-2201:35seancorfield(it doesn't mean the project is relying on Leiningen using tools.deps)#2019-05-2201:35alexmillerwell what are you calling clojure-env for?#2019-05-2201:35alexmillerI assume the locations of the deps.edn files#2019-05-2201:36seancorfieldRight, so that those can be passed back in to tools.deps...#2019-05-2201:36alexmillerthere are 3 - install (goes away), ~/.clojure (deterministic based on env vars), current directory#2019-05-2201:36alexmillerthe first one is the hard one#2019-05-2201:37seancorfieldAh, and you're going to fold the system/install one back into the library, right?#2019-05-2201:37alexmilleryes#2019-05-2201:37alexmillerthis also removes the biggest pain in the ass in the installer which is also finding that path#2019-05-2201:38alexmillerwell, maybe it's the path to the jar itself#2019-05-2201:38seancorfield(which is what I had originally done with boot-tools-deps -- baked in a copy of it from the Clojure tools repo 🙂 )#2019-05-2201:38alexmilleryeah. doesn't even need to be a file, could just hardcode parts of it#2019-05-2201:38hlollithis project in question is Overtone, I wanted to add support for tools.deps, so I had to add badegion to extract the native deps. Badegion is what's calling tools.deps.alpha namespace. But in discssion with the maintainer of badegion, it seems I can do all that without deps.edn being in the jarfile.#2019-05-2201:39alexmillerthe root deps.edn is just:#2019-05-2201:39alexmiller{
:paths ["src"]
:deps {
org.clojure/clojure {:mvn/version "1.10.0"}
}
:aliases {
:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.6.496"}}}
:test {:extra-paths ["test"]}
}
:mvn/repos {
"central" {:url ""}
"clojars" {:url ""}
}
}#2019-05-2201:39seancorfieldOK, once tools.deps bakes that in and all the tooling has rev'd to depend on that version and everyone has updated to the latest versions of those tools, we will all be set! 🙂#2019-05-2201:41seancorfieldI am now remembering this discussion -- but it was quite a while ago and I guess I'd forgotten that it was seriously being considered. Good to hear @alexmiller#2019-05-2201:41alexmilleryou can also just use tools.deps directly, hard-code your own deps.edn root with the stuff you need from that, and not ever call clojure-env#2019-05-2201:42alexmillerwhich is admittedly more work, but is also something that does not depend on anyone else doing anything#2019-05-2201:42seancorfieldYeah, but I think tooling has all evolved to assume that calling clojure-env is the "correct, supported way" to get that information :rolling_on_the_floor_laughing:#2019-05-2201:42alexmillerbleh#2019-05-2201:42alexmillerevil#2019-05-2201:42alexmillerI'll just change it to return "thppppt"#2019-05-2201:44seancorfieldlein-tools-deps has actually copied in that function https://github.com/RickMoynihan/lein-tools-deps/blob/master/src/lein_tools_deps/env.clj#L32#2019-05-2201:46hlolli@seancorfield amazing, thanks, my error is actually coming from right there but not tools.deps.alpha 😄#2019-05-2201:46hlollithe windows cmd is hard to read#2019-05-2201:46alexmillerwell I can't fix that :)#2019-05-2201:46hlolliexactly 🙂#2019-05-2201:46seancorfieldIt had been a while since I last looked at the source of Rick's project... At least boot-tools-deps is archived now so folks will be discouraged from using it 🙂#2019-05-2215:07WhittlesJrFor a maven repo with a self-signed SSL cert, lein had me add the path to my system's ca-bundle.crt in profiles.clj / :user / :certificates. What do I do with tools.deps to make that work?#2019-05-2215:10WhittlesJrUp until now I've been able to forget about our internal maven server and just use :git/url, but there's a java project out there (https://github.com/entropia/libsocket-can-java) that never released to maven central that we had to build and host ourselves.#2019-05-2215:30alexmillernothing in tools.deps that is similar. what happens if you try it?#2019-05-2216:09WhittlesJrerror in process sentinel: nrepl-server-sentinel: Could not start nREPL server: Error building classpath. Failed to read artifact descriptor for de.entropia:libsocket-can-java:jar:0.3.0
...
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
It might be my OS configuration, I'm looking into it.#2019-05-2216:31alexmillerusually you would update cacert in your Java installation for this I think#2019-05-2216:35alexmillerthat's probably enough to google for how to do it#2019-05-2220:07borkdudeI’m trying to get rid of brew in a Mac environment and trying the install: illegal option -- D
Any workarounds for this not involving brew?#2019-05-2220:09borkdudethe reason is that brew from scratch is terribly slow on CI#2019-05-2220:11ghadiyou can cp + chmod instead of install#2019-05-2220:12ghadiyou can mimic the "spirit" of the install script#2019-05-2220:12borkdudeyeah, that works 🙂#2019-05-2220:14alexmilleryou can also just look at what the brew script does (which is largely to put the tar in a place and run its own variant of the install script)#2019-05-2220:14borkdudethat works too#2019-05-2220:15alexmillerit basically downloads https://download.clojure.org/install/clojure-tools-1.10.0.442.tar.gz, expands, cds into clojure-tools, and runs install.sh prefix from it#2019-05-2220:16borkdudethat sounds easy enough, ok#2019-05-2220:17alexmillerI guess it matters where it downloads and does that stuff and that's the env brew provides#2019-05-2220:18alexmillerultimately there is clj/clojure, a jar file, deps.edn file, etc and it's just a matter of putting those all in the right places and pointing at each other#2019-05-2221:43borkdudegot it working now#2019-05-2221:49borkdudeclojure now installs in 0 seconds in CI instead of 5 minutes 😉#2019-05-2223:09hlolliIs there a way to set warn-on-reflection true globally in tools.deps, some trick that someone has found out?#2019-05-2223:11hlollinevermind, I should get used to just keeping it in my clojure files#2019-05-2223:19alexmillerI think there’s a system property for that? Not at a computer to look it up.#2019-05-2223:21hlollioff-topic, I'm in the process of trying to get standalone jar to run, is it safe to ignore loop/recur symbol warnings that end with Auto-boxing loop arg: some-symbol ?#2019-05-2223:25alexmillerDepends whether you care about perf#2019-05-2223:31hlolliok, so it's possible to remove these reflections, my misunderstanding#2019-05-2223:32hlollithis looks about right? :jvm-opts ["-Dclojure.compile.warn-on-reflection=true"]#2019-05-2223:33hlollithere's a ticket about this https://clojure.atlassian.net/browse/CLJ-2328#2019-05-2223:33alexmillerLooks good as far as I remember off the top of my head#2019-05-2223:34alexmillerOh yeah, so then no :)#2019-05-2223:34hlolliI'm running compile/compile, and I only hear crickets#2019-05-2300:17seancorfield@hlolli, for now, you have to have (set! *warn-on-reflection* true) in each source file. I think it's a good habit to get into.#2019-05-2302:15hlollihow would a subdirectory fit with git/url, thinking of https://github.com/bhauman/rebel-readline/tree/master/rebel-readline#2019-05-2302:28alexmilleryou can specify :deps/root as a subdir in which to find deps.edn/pom.xml in the context of a git lib#2019-05-2302:29alexmillerdoesn't look like that project needs that though? there's a deps example in the readme#2019-05-2308:03ewenIs there a way to add clojure$slim (slim classifier) as a dependency and exclude the clojure dependency that comes with tools.deps ? Adding clojure$slim to deps.edn results in both clojure and clojure$slim to be on the classpath. Looks like a consequence of the recent classifier change ?#2019-05-2308:30dominicmAh, because it comes in from the "system" deps.edn, that's interesting.#2019-05-2308:30dominicmI guess the original :classifier design was better for this case.#2019-05-2311:16alexmillerno not possible right now. Classifier deps really are treated as separate deps and because Clojure is included by default, there is no way to remove it#2019-05-2311:18alexmillerHadn’t ever thought of this re slim (almost no one uses it)#2019-05-2314:44lilactownis there a way to determine a proper “groupId” from a deps.edn? not really right?#2019-05-2315:24alexmillerYou mean for deps or for the project itself?#2019-05-2315:29lilactownfor the project itself#2019-05-2315:32alexmillerthen, no#2019-05-2315:32lilactownnot super important, I’m just writing a utility to add a dep to a project via command line. ran into where:
- plum add org.clojure/clojure "1.10.0" appends org.clojure/clojure {:mvn/version "1.10.0"}
- plum add "some-long-sha" right now adds as
clojure/clojure {:git/url "" :sha "some-long-sha"}
#2019-05-2315:33lilactownthe pragmatic thing to do would be to check for a pom.xml in the repo, I suppose. or just not worry about it#2019-05-2315:34alexmillerwell using clojure as a git dep like that won't work (as it's half java and must be compiled into an artifact)#2019-05-2315:35lilactownah, fair point#2019-05-2315:36lilactownI think the problem exists still as: I want to use the latest version on git of a library that I previously was consuming through maven. with my naive approach, a user might end up with two different identifiers in their deps.edn for the same lib#2019-05-2315:37lilactownwhich, btw, what would the behavior be in that case? last one wins?#2019-05-2315:39alexmillerright now, it will error I think#2019-05-2315:39alexmillerit is, however, actually possible to resolve as maven artifacts include their git commit info at build time, making it possible to resolve the "newer" one via git commit comparison#2019-05-2315:39alexmillerI have not actually done that work yet, but have done enough research to believe it's feasible#2019-05-2316:57lilactownah neat. thanks for the info!#2019-05-2317:51WhittlesJrYesterday I was asking about adding a self-signed SSL cert with tools.deps, and I did some research on it. I'm on NixOS, which is a functional / immutable OS, and as such its Java trust store is also immutable (read-only), so I can't just imperatively add the cert to the trust store using keytool. Unfortunately right now there's not a configuration option in NixOS to do this either. Am I SOL?
https://stackoverflow.com/a/49423656/2015090#2019-05-2400:21mynomotoI generated a pom for my project using -Spom so I could use dependency-check-maven to find vulnerabilities on dependencies. But since the way maven and tools.deps solve dependencies I got discrepancies on the dependency tree calculated by maven and by clj. Could -Spom return all dependencies including indirect ones or have another command that could do that?#2019-05-2400:34alexmillerthe trees are really different because mvn and clj resolve dependencies differently. which one do you actually care about?#2019-05-2400:34alexmilleryou can grab the actual full set of deps clj is resolving (as data) by looking at the .libs file that's generated#2019-05-2400:35alexmillerif you do clj -Sverbose you'll see a cp file like .cpcache/723570742.cp - if you replace .cp with .libs, that's the file#2019-05-2400:36alexmillerthat's an edn file, so you could suck it in with clojure and manipulate it however you need#2019-05-2400:41mynomotoI care about the clojure one, since that will go to production, but the tool to check dependencies uses the pom information.#2019-05-2400:43mynomotoSo reading the .libs file and generating a pom.xml with all dependencies should work.#2019-05-2401:18seancorfield@alexmiller Does that mean that deps-based projects that use clj -Spom to generate/maintain a pom.xml that is published to Clojars can cause projects that use them to get the "wrong" dependencies?#2019-05-2401:29alexmillerthe generated pom should have only the same top level deps#2019-05-2401:30alexmillermy understanding of the request above is that he is scanning the full transitive dep tree, which may indeed be different between clj and (mvn or lein)#2019-05-2401:34mynomotoThis is accurate.#2019-05-2401:31alexmillerthese differences are presumably mostly same libs but different versions, but in cases where transitive deps change, could be different deps#2019-05-2401:33alexmillerin theory, that may be an issue. in practice, it seems to rarely matter, perhaps because most libs are subsequently used as someone else's lib, and then lose all control over their dep versions anyways.#2019-05-2401:40seancorfieldThanks for the clarification!#2019-05-2402:12mynomotoI have mixed feelings about my experience with the classpath differences today. lein and boot do the same as Maven in resolving transitive dependencies so this is a tools.deps only problem. If -Spom is supposed to create a classpath for libraries it's doing fine with the caveat that if you want to analyze the tools.deps library classpath using the generated pom using Maven tooling you can get results that are not real, but as Alex pointed out, in the end is the application classpath that matters. In the end I think that having an easy way of generating a pom.xml that will create the same classpath as tools.deps is important because of tooling that already exists for Maven.#2019-05-2402:20mynomotodependency-check-maven plugin pointed several high/critical vulnerabilities on dependencies used by pedestal and buddy mentioning only libraries that I think are popular in the clojure world. I had no time to check if those could be exploited. This is on the latest release of both libraries. The dependencies have new releases not affected by the vulnerabilities.#2019-05-2402:30alexmillermy limited experience with dependency-check-maven and the underlying stuff is that the matching is very bad and often highlights libraries that have nothing to do with the CVEs it associates with it#2019-05-2402:33alexmilleran example is https://github.com/jeremylong/DependencyCheck/issues/1788 which flagged data.zip afaict because it had "zip" in the name and that matched a totally unrelated "zip" project. I dug into the code just enough that I have a really low confidence in the results of this tool. So, fyi.#2019-05-2402:34mynomotoHaving spent a fun afternoon reading cves, looked legit. Cves were very specific on coordinates and versions affected.#2019-05-2402:35alexmillerif you have a list of issues, would be great if you could just drop it somewhere so we could do necessary stuff to improve things#2019-05-2402:35alexmillerissues on specific tools would be better, but if you've at least got a list, that would make it faster for someone else to log if you don't have time#2019-05-2402:37alexmilleron the suggestion re generating a pom, the goal was to create a pom that was similar to the deps.edn, which meant just the top level deps. Stating all lower level deps in the pom would be (conceptually) much different. That's more than I'm planning to take on for -Spom (which itself may get pulled out of tools.deps/clj and put into a standalone tool)#2019-05-2402:38alexmillermaking a tool that used tools.deps and a deps.edn and did the dependency checks directly without involving Maven would be even better#2019-05-2801:27Richard GouldWe've written something internally that we've been using for a few months to do this. Have just gotten clearance to open source it, hope it's useful. Available now at https://github.com/ROKT/check-deps
@U9Y2R7YAV#2019-05-2402:51mynomotoI understand the trade-offs of generating a pom similar to what other tools generate. But that comes with the inconvenience of having different classpaths on different tools because the algorithm used for transitive dependencies is different and the results are different than one would expect. While a tools.deps specific dependency check tool would be indeed better that would mean another tool that someone would have to maintain. Using one that already exists and is maintained has lots of advantages too. I will rerun the analysis without my dependencies overrides and report on each tool tomorrow.#2019-05-2402:57alexmillerthanks, that would be great#2019-05-2403:42mynomotohttps://github.com/pedestal/pedestal/issues/619#2019-05-2403:51mynomotohttps://github.com/funcool/buddy-core/issues/66#2019-05-2404:07alexmillerlooks like good reports, thx!#2019-05-2409:33weiis there a good way to share code between two tools.deps projects living in the same git repo?#2019-05-2409:39dominicm@wei use a local dependency when developing them, I think that's what you're asking.#2019-05-2409:40weiah thanks for the tip. reading https://clojure.org/guides/deps_and_cli#_using_local_libraries now#2019-05-2809:41thheller@alexmiller I think a shadow-cljs user stumbled into a bug in tools.deps regarding :exclusions of libs. not sure if intentional.#2019-05-2809:41thheller{:deps
{org.clojure/clojure {:mvn/version "1.10.0"}
thheller/shadow-cljs {:mvn/version "2.8.37"}
io.pedestal/pedestal.service {:mvn/version "0.5.5"}
}}#2019-05-2809:41thhellershadow-cljs has a dependency on ring-core but excludes clj-time#2019-05-2809:42thheller[ring/ring-core "1.7.1"
:exclusions
;; used by cookie middleware which we don't use
[clj-time]]#2019-05-2809:42thhellerpedestal.service also depends on ring-core but the exclusion from shadow-cljs is applied to its dep#2019-05-2809:43thhellerso clj-time is missing#2019-05-2810:19rickmoynihanDo any of the tools-deps tools for generating an uberjar have support for something like lein’s :uberjar-merge-with?#2019-05-2810:20dominicmPack avoids conflicts altogether with a classloader.#2019-05-2810:21rickmoynihanThe thing is, I need to merge edn files#2019-05-2810:22rickmoynihane.g. data_readers.edn but in particular duct_hierarchy.edn#2019-05-2811:03rickmoynihanok actually avoiding the conflicts and using a classloader would probably work… I guess I can just walk the resources and merge them myself.
Which of the uberjar options will use a classloader? I’m guessing capsule?#2019-05-2811:05rickmoynihanhmmm… infact I probably don’t need to merge them myself… duct must be doing this already to work at the repl.#2019-05-2811:07rickmoynihanindeed it does: https://github.com/duct-framework/core/blob/c5fb68ac3539248942c50cdbba5a2ca63735ee3c/src/duct/core.clj#L42-L44 I’d literally just written those same three lines.#2019-05-2811:17rickmoynihanok I can confirm pack.alpha capsule just works ™ thanks#2019-05-2811:33rickmoynihanalso capsule is awesome 😀#2019-05-2812:54dominicmCapsule is very cool :)#2019-05-2812:55dominicmPack currently has a no conflict policy so that you don't have an option to shoot yourself in that respect.#2019-05-2813:54rickmoynihan👍 a good policy to have#2019-05-2813:56rickmoynihanwhat is the purpose of -e in pack.alpha? Why not just an alias with the extra-paths there? Or is it different somehow?#2019-05-2813:56rickmoynihanIs there any extra meaning in
> Add directory to classpath #2019-05-2814:23dominicm@rickmoynihan no extra meaning. That flag will continue to exist so that I can put directories generated with mktemp into the class path.#2019-05-2814:24rickmoynihanahh makes sense thanks#2019-05-2814:25dominicmThe "for building" is probably a bias in my writing for my particular use case.#2019-05-2814:27rickmoynihanyeah — but that use case makes sense 🙂 the docs could possibly benefit from a clarification as to that purpose… but no biggie 🙂#2019-05-2814:27rickmoynihanit is essentially the same as :extra-paths but more suited for scripting/api usage in wider build processes#2019-05-2814:36dominicmI'm terrible at writing documentation :)#2019-05-2817:33rickmoynihan@dominicm: incidentally it seems that pack.alpha doesn’t seem to honor :extra-paths in other aliases you supply during the pack phase. I’m guessing this is a bug? Though I could have my wiring wrong.#2019-05-2817:33dominicmHmmm, that would be a bug, yeah.#2019-05-2817:34dominicmI thought I tested that though#2019-05-2817:35rickmoynihanok I’ll open an issue for it — and also check it’s a real problem.
Issue filed here: https://github.com/juxt/pack.alpha/issues/36#2019-05-2818:05Daniel HinesReposting noob cli question from #beginners:
Is there a way to create a “default profile” for the clj cli? Such that whenever I run clj, certain deps are always merged in?#2019-05-2818:05Daniel Hineslike a ~/.deps.edn file or something?#2019-05-2818:11rickmoynihanyou can include custom (private) aliases in ~/.clojure/deps.edn but as far as I know you always have to opt in to aliases#2019-05-2818:11rickmoynihantools.deps prefers to be explicit#2019-05-2818:12didibusYes, put a deps.edn file here $HOME/.clojure#2019-05-2818:16didibusNah, you can just add normal deps#2019-05-2818:16didibusAnd they'll always be included#2019-05-2818:16didibusThat's how Clojure itself gets loaded#2019-05-2818:17Daniel HinesDumb question from a non-Java person: what is an “alias”? https://clojure.org/reference/deps_and_cli#_aliases doesn’t really explain that.#2019-05-2818:17Daniel HinesAlso, -S seems to be missing from that list of possible aliases.#2019-05-2818:17didibusThere are 3 deps.edn. The install one, the one in your home directory. And optionally, your project one. The :deps map gets merged from install to home to project. In case of conflicts, the later ones override the prior ones.#2019-05-2818:19didibusAliases are like dynamic modifications#2019-05-2818:20didibusNormally, the classpath, that is, the path to all libraries available to your program is defined by the :deps map, and the :paths vector, and a few others.#2019-05-2818:21didibusBut, maybe on occasions, you want to start a program or a repl with more or less libraries included#2019-05-2818:21didibusThat's what aliases let you do#2019-05-2818:21didibusSo you can define sets of dependencies and name them with an alias.#2019-05-2818:22didibusNow, from the command line, you can choose to include some of those additional sets by adding the alias for it#2019-05-2818:23Daniel HinesThat’s a very helpful explanation @didibus, thanks. So including aliases merges the options contained in those aliases?#2019-05-2818:23didibusYa#2019-05-2818:24Daniel HinesYou said it’s possible exclude as well? How would that be accomplished, since it’s a merge?#2019-05-2818:24didibusPretty much. Now the way things are "merged" can vary depending in the type of alias and config.#2019-05-2818:25didibusWell, you can't exactly exclude. So you'd be better off thinking of them more as inclusions#2019-05-2818:26didibusBut for example, you can specify :override-deps which will override versions#2019-05-2818:26didibusOr :classpath-overrides which overrides the path to a lib#2019-05-2818:28seancorfield@d4hines As an example of what you can do with ~/.clojure/deps.edn, here's mine https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#2019-05-2818:29seancorfieldThat has some usage notes as comments as well. Plus the README in that repo explains in more detail...#2019-05-2819:25Daniel HinesThis is super helpful, thanks!#2019-05-2818:33didibusI'm not sure if you could classpath-override to a null dir#2019-05-2818:33didibusOr a dir which doesn't have the dependency#2019-05-2818:33didibusAs a weird way to exclude#2019-05-2818:33didibusBut I wouldn't recommend it#2019-05-2818:34didibusBetter design it so aliases are additions or overrides, but not removals#2019-05-2818:39didibusAlso, I'm not sure what you mean by -S, that's just a prefix for other cli options. It's not related to aliases.#2019-05-2917:55rickmoynihanwhen using :git/deps does :deps/root need to specify a directory containing a deps.edn file or can it be a path to a deps.edn file named something arbitrary, like foo.edn?#2019-05-2917:55rickmoynihanI’m guessing it must be a directory#2019-05-2918:08dominicmYeah, directory#2019-05-2918:28rickmoynihanI’m assuming it’s not possible to override just the :deps/root for a :git/dep in aliases? Use case is sharing a single :git/url and :sha across multiple aliases that parameterise just the :deps/root.
I’m guessing that to do this I would need to duplicate the :sha and :git/url as :extra-deps in each alias.#2019-05-2919:15dominicmYeah, duplication#2019-05-2919:15dominicmI'd write a tool to manage it for me#2019-05-2919:35holyjakIs there a trick to not need to repeat the version number here: {:deps {org.apache.lucene/lucene-core {:mvn/version "8.1.1"}
org.apache.lucene/lucene-queryparser {:mvn/version "8.1.1"}
org.apache.lucene/lucene-analyzers-common {:mvn/version "8.1.1"}} ?#2019-05-2919:47seancorfieldNope.#2019-05-2919:47seancorfieldUnless there's a bundled artifact you could depend on that would bring in all those pieces.#2019-05-3009:42dm3How do I specify repository mirrors with deps.edn?#2019-05-3009:49rickmoynihanHow are people deploying artifacts to private s3 buckets with tools.deps? Are people using mvn deploy, or scripting up pomegranate aether with s3-wagon-private somehow?#2019-05-3009:56rickmoynihanpomegrante’s aether deploy looks like it might be easy enough… though not sure how to wire in the s3 wagon…#2019-05-3009:57rickmoynihanIt looks like it might be automagically done#2019-05-3010:32rickmoynihanok it looks like it’s pretty easy to get pomegranate deploy with s3 wagon working in a simple clj script#2019-05-3011:00dominicm@rickmoynihan write that down somewhere for others!#2019-05-3011:04rickmoynihanWill do. It looks like it basically just works. I’m sure it’d be easily extractable into an aliasable task, though I haven’t thought too much about how it should work in that form.#2019-05-3011:05rickmoynihanRight now I essentially want to programmatically generate a bunch of very similar artifacts that differ only in terms of one or two files and their artifact id.#2019-05-3011:08rickmoynihanThe higher level problem I’m looking to solve is we have a repo that contains our own css framework, and we want to package client themes from a single place but consume them downstream as essentially webjars. So each clients theme will typically differ only by their theme.css#2019-05-3011:09rickmoynihanso I’ll likely solve that problem first, then look at extracting something reusable.#2019-05-3019:55mnewtIs there a way to specify a dependency for a leiningen project? (i.e. project.clj is the only manifest)#2019-05-3020:41ballparkIs this what you're looking for: https://github.com/RickMoynihan/lein-tools-deps ?#2019-05-3020:49mnewt@ballpark Thanks for that. That is useful but unfortunately not quite what I’m looking for. I was hoping that there was a way to pull in an existing leiningen project without modifying it. Is there a library or something that can resolve dependencies from a project.clj file or is the only way to do that to run leiningen itself? My use case is I would like to include specific versions of projects directly from github.#2019-05-3021:10seancorfield@mnewt Petition the maintainer to add deps.edn 🙂 You could always override the deduction, specify the manifest as :deps and then provide the transitive dependencies yourself in your deps.edn file.#2019-05-3021:12mnewt@seancorfield Thanks, that makes sense#2019-05-3112:48dharriganIf I have a configuration in my $HOME/.clojure/deps.edn file, thus: {:extra-deps: {foo-bar {:local/root "libs/foo}}}, it doesn't appear that resolution happens relative to the $HOME/.clojure" directory, I have to put the full path in, i.e., ....{:local/root "/home/foo/.clojure/libs/foo"#2019-05-3112:50dharriganIs my thinking correct on this? No way for the resolution to happen relative to where the deps.edn file is?#2019-06-0312:41rickmoynihanI think that’s correct, as things will get more complex when you have transitive git deps etc.#2019-06-0312:54dharriganthanks for the confirmation 🙂#2019-05-3122:15sparkofreasonI still have this exception running clojure from WSL. I've confirmed that ssh-agent is running and configured. com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available: ssh-agent:jna,ssh-agent:nc Does anybody know the solution?#2019-05-3123:06hiredmanYou need some native code to talk over Unix sockects on the jvm, which is how you interact with ssh-agent. The jsch library might not be able figure out the native code required for wsl#2019-06-0100:28sparkofreasonI assume people have this working, and there's just something about my particular WSL install (pengwin) that is an issue.#2019-06-0102:06TravisDoes anyone know if there is a way to override where tools-deps looks for the .m2 directory ? For instance with Maven you can do something like
MAVEN_OPTS=-Dmaven.repo.local=/cache/.m2
#2019-06-0111:01alexmillerYou can set :mvn/local-repo at the root in deps.edn#2019-06-0222:17Travis@U064X3EF3 is there a way to set this at the command line for override in some build env?#2019-06-0222:22alexmillerYou can use -Sdeps with an in-line deps.edn to add it on the command line#2019-06-0111:08pezHello. I am trying to get the Calva jack-in story to be told nicely on Windows. Can I assume that powershell is used for cli/deps?#2019-06-0111:09pezAlso I am a bit confused about when the command line tool is named clj and when it is named clojure.#2019-06-0112:19sogaiu@pez in *nix and macos isn't one just a wrapper to use the other along with rlwrap? have a look inside the files?#2019-06-0112:23pezYes, checked the contents on my Mac now. clj wraps clojure in rlwrap. So it is clojure I should use for jack-in then. Thanks!#2019-06-0117:49yuhanI tried unsuccessfully to install the cli on Windows a while back too, to my understanding the official guide at https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows only installs a PowerShell module in ~/Documents#2019-06-0117:49yuhanbut I could not figure out how to get from that to a clj / clojure executable on the PATH#2019-06-0317:11rickmoynihanI’m running a clojure CLI tool inside travis, and one of the deps is in a private github repo; and it dies with an error like:
Error building classpath.
The build itself is itself inside a private repo too, and travis seems to be able to clone that ok.
Any ideas what I might need to do?#2019-06-0317:16dominicm@rickmoynihan I would guess ssh-add or similar#2019-06-0317:16rickmoynihanyeah#2019-06-0317:16dominicmIs an agent running?#2019-06-0317:17rickmoynihanI would’ve expected travis to already have that setup though, given that one of the first things it does is clone a private GH repo#2019-06-0317:17dominicmYeah. I wouldn't be certain it's using an agent#2019-06-0317:17rickmoynihanfair#2019-06-0317:18dominicmThe other thing is, are you using gitlab?#2019-06-0317:18rickmoynihangithub#2019-06-0317:36dominicmOkay, then the other bug I'm thinking of doesn't apply#2019-06-0317:48alexmillerclj uses jgit/jsch (not git) so it's definitely possible to have git work on the initial clone and then tools.deps fail to clone a dep repo#2019-06-0317:50alexmillerthere are a number of deficiencies in the jgit/jsch stuff (we're on a somewhat old version now too). definitely needs some attention and we're considering either using a newer version or calling out to git or possibly making both available#2019-06-0320:16WhittlesJrDoes tools.deps support loading native .so libs via JNI? With leiningen, I had:
:resource-paths ["lib"]
:native-path "%s/lib"
:jvm-opts [~(str "-Djava.library.path=lib/:" (System/getenv "$LD_LIBRARY_PATH"))]
And that did the trick. I've tried a few different things but can't get this working with tools.deps (I get an UnsatisfiedLinkError.)#2019-06-0320:17alexmillerno, this is not yet supported#2019-06-0320:18alexmillertools.deps can add paths to the classpath and use jvm opts, but it doesn't have support for unpacking or loading native libs#2019-06-0320:19WhittlesJrDang, alright. Thanks.#2019-06-0320:29dominicm@alex.joseph.whitt what lib is this? Most of the ones I've seen unpack themselves.#2019-06-0320:31WhittlesJrit's built from this: https://github.com/entropia/libsocket-can-java/blob/master/jni/cansocket.cpp#2019-06-0320:33alexmillerlein has a thing I think that unpacks native libs#2019-06-0320:33alexmillerI don't know the details#2019-06-0320:36WhittlesJrFor now I'll probably try to follow this approach: https://stackoverflow.com/a/28012737/2015090#2019-06-0320:37WhittlesJrIt would be real nice if this was just built in#2019-06-0322:01jeroenvandijk@rickmoynihan Almost bedtime, but you can add a user's git keys with circleci and this allows you to use private libs (assuming they are accessible with those keys). I can have a look tomorrow if necessary#2019-06-0409:55martinklepschcurl -O
clj -Sdeps '{:deps {org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu #:local{:root "mxnet-full_2.11-linux-x86_64-cpu-1.4.1.jar"}}}'
Error building classpath. 1 problem was encountered while building the effective model for org.apache.mxnet:mxnet-core:[unknown-version]
[FATAL] Non-resolvable parent POM for org.apache.mxnet:mxnet-core:[unknown-version]: Could not find artifact org.apache.mxnet:mxnet-parent:pom:INTERNAL in central () @
Is this supported and a bug or am I doing something wrong here?#2019-06-0411:17alexmillerYou might need to add a maven repo if that parent pom isn’t in maven central#2019-06-0414:51martinklepschThe weird thing is that this works:
clj -Sdeps '{:deps {org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu {:mvn/version "1.4.1"}}}'
#2019-06-0414:53alexmillerwe do have a known issue that repos defined in the pom don't get used and that may be the issue here#2019-06-0414:54martinklepschbut then the above should also not work, right?#2019-06-0414:56alexmillerthe last example there is a different path and relies on the maven repo to resolve artifacts (https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-46 is the issue I'm thinking of)#2019-06-0414:56alexmillerwhen you load a jar dep, tools.deps is reading the model and resolving the deps#2019-06-0414:56alexmillerI think it's in the tools.deps pom reader where more needs to be done#2019-06-1009:11martinklepschI'm not sure if TDEPS-46 s whats going on here. Just let me know if you want a separate JIRA to document this failure case#2019-06-1012:40alexmilleryou could leave a comment on that ticket so it can be checked and spun out if needed#2019-06-1016:14martinklepschIt seems I can no longer comment: could you give permission to <mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>#2019-06-0411:30dm3are repository mirrors supported?#2019-06-0412:13alexmillerI don't think so? I think that's still a todo#2019-06-0412:17alexmilleryeah, that's not supported atm and I don't actually see a ticket for it, although that would be fine if you wanted to file one https://clojure.atlassian.net/servicedesk/customer/portal/1#2019-06-0415:38WhittlesJr@dm3 I've got repository mirroring working, if we're talking about the same thing. My deps.edn config includes:
:mvn/repos {"central" {:url ""}}
And my ~/.m2/settings.xml:
https://gist.github.com/WhittlesJr/3422e3374c0393188c1321bbd7605bd5
We're using Nexus 3 for our maven server ("my-mirror"). Caches everything and works like a charm.#2019-06-0415:42WhittlesJr(I had to use http because I couldn't add a self-signed certificate to the java trust store in my particular case)#2019-06-0415:44WhittlesJrI suppose that having all that config in settings.xml breaks repeatability though, so a pure tools.deps solution would be very nice indeed.#2019-06-0415:45WhittlesJr(Er, maybe it is repeatable after it's all cached? Perhaps you could drop the settings.xml after that point and it would still download what it needed from "my-mirror." I'm not sure.)#2019-06-0415:51alexmillerwell that's not mirroring, that's just replacing the repo#2019-06-0415:51alexmillerthat should work#2019-06-0415:52alexmillermaven actually has support for mirrors (or maybe the term is different, I can't remember)#2019-06-0415:52alexmillermirrors being duplicate repos that can serve the same content#2019-06-0416:02WhittlesJrYeah I think I've seen the word "mirror" used for both meanings so I'm not sure what term to use. It is at least called "mirrors" in settings.xml (the <mirrors> section). On first request, Nexus will download and cache the needed artifacts from maven central or clojars, and then the client downloads everything from Nexus. Is that different from what you're talking about?#2019-06-0416:03WhittlesJrI think I've also heard it called "proxying"#2019-06-0416:05WhittlesJrBut that's also confusing because "proxying" usually refers to using an http proxy#2019-06-0416:13WhittlesJrMaybe what I'm doing is just one species of mirroring: https://maven.apache.org/guides/mini/guide-mirror-settings.html
(third bullet point)#2019-06-0419:06dm3yeah, you’re overriding the repo#2019-06-0419:06dm3mirror is another layer#2019-06-0513:40sveri@alexmiller Hey, I just came back from a longer break and noticed tools-deps is finally available on windows, which is pretty awesome, thanks 🙂#2019-06-0513:50alexmillergive it a whirl#2019-06-0513:55sveriYea, I just did and after I allowed myself to run unsigned scripts in the powershell installation worked and I can run it#2019-06-0513:55sveriJust about to experiment with it finally#2019-06-0517:44markxHi all, why is sha required for a :git/url? Is it possible to automatically use master branch?#2019-06-0517:45hiredmanyou can't cache a reference#2019-06-0517:46hiredmanclj startup performance is based building a classpath once and caching it for the same deps.edn file#2019-06-0517:50markxAh makes sense. It doesn’t have a record of the local version.
I was just hoping to achieve some function like npm or pip or other package mangers.#2019-06-0517:52seancorfield@markx If you really want to be able to add libraries from master, you can do something like this https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L106-L119#2019-06-0517:56markxI still need to wire my brain to connect the repl and the cli tool. I understand the code but don’t know how to do it on cli.#2019-06-0517:58markxWhat I really really want is something like clj install xxx then I can just run xxx from my shell.#2019-06-0517:58markxWhat I really don’t like is to build or download a jar and manually put it into PATH.#2019-06-0517:59lilactownhey markx. a thing I'm building might help#2019-06-0517:59lilactownis this for installing libraries, or installing applications?#2019-06-0517:59markxIdeally for both, but mostly for applications.#2019-06-0518:02lilactownhttps://laughing-banach-af1115.netlify.com/#2019-06-0518:03lilactownit doesn't have direct support for installing global aliases right now. but it does have a plum add command that adds a dependency to a local deps.edn#2019-06-0518:12dominicm@U4YGF4NGM do you have a plan to preserve whitespace when you do this?#2019-06-0518:13lilactownit should preserve whitespace#2019-06-0518:13lilactownwell, it formats it using cljfmt#2019-06-0518:14lilactownwhich I suppose is not the same 😅#2019-06-0518:14dominicmI would not use this, because that will lose comments.#2019-06-0518:14lilactownno, it will preserve comments. that is why I moved to using cljfmt / rewrite-clj in the first place#2019-06-0518:15lilactowngo ahead and give it a shot#2019-06-0518:15dominicmOh, you're using rewrite-clj directly too :)#2019-06-0518:15dominicmGood. That's what I was going to suggest if you didn't have an idea.#2019-06-0518:15dominicmBut I was hoping you'd found a novel approach I didn't think of :)#2019-06-0518:15dominicmI'll look through the code to compare.#2019-06-0518:16lilactownyep 😄 cljfmt is there for formatting, and then rewrite-clj for actually doing the addition:
https://github.com/Lokeh/plum/blob/master/clj/src/plum/add.clj#L56#2019-06-0518:17dominicmI had some users get quite annoyed that they lost their carefully aligned arguments when I first added this, just fyi#2019-06-0518:18lilactownthat makes sense. I wish there was a way to do preserving modifications of EDN better#2019-06-0518:19dominicmI spent some time on it in ednup#2019-06-0518:19dominicmThe goal was to make a clojure map you could assoc without losing formatting#2019-06-0518:20lilactownI wonder if we could format the modification and then assoc that using rewrite-clj to not mess with the rest of the file?#2019-06-0518:57dominicmWhat I did in ednup was look at the previous key, and match the whitespace based on that#2019-06-0518:11markxThis is a nice tool. I read about it on clojureverse.
IIUC, it provides cli commands to avoid manually editing the deps.edn file. But it doesn’t solve all my pains. I hope it can go one step further.
Do you plan to support installing global aliases?#2019-06-0518:12lilactownit could be added. if it would help you, create an issue on github.#2019-06-0518:13lilactownI would also look over a PR if you're interested in contributing 😄#2019-06-0518:40markxYeah I’m interested. I’ll try. But don’t wait for me. 😛#2019-06-0607:07sveriHow do I start a repl in a given namespace? I have not seen anything in the documentation about it.#2019-06-0608:21sveriI figured I could use something like this:
:aliases
{:cursive {:main-opts ["-e" "(load \"de/sveri/getless/user\")"
"-e" "(de.sveri.getless.user/reset)"]}}
It works, but is it the right way to use tools-deps?#2019-06-0612:10alexmillerYou would need to do something like this#2019-06-0612:11alexmillerYou could simplify to one expr by using requiring-resolve (added in 1.10)#2019-06-0612:13alexmiller((requiring-resolve ‘de.sveri.getless.user/reset))#2019-06-0612:32sveri@alexmiller Thanks, that works and looks better#2019-06-0612:41alexmilleralthough double parens always looks weird#2019-06-0622:38holyjakIs :local/root supposed to work for pointing to a local lein project (with a pom.xml)? Or only deps projects? Thanks! (It doesn't for me)#2019-06-0622:58alexmillerit should work for deps.edn and pom.xml#2019-06-0622:58alexmillerin what way does it not work?#2019-06-0709:44holyjakE.g. REPL does not see the code from that project: $ clj -A:test
Clojure 1.10.1-beta2
user=> (require '[clj-concordion.core :as cc])
Execution error (FileNotFoundException) at user/eval1 (REPL:1).
Could not locate clj_concordion/core__init.class, clj_concordion/core.clj or clj_concordion/core.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
with deps.edn including {:aliases
{:test
{:extra-deps {clj-concordion {:local/root "/Users/me/Projects/clj-concordion"}
;clj-concordion {:local/root "/Users/me/Projects/clj-concordion/target/clj-concordion-0.0.5.jar"} (the .jar works) and $ head ~/Projects/clj-concordion/pom.xml
<?xml ...><project ...>
<modelVersion>4.0.0</modelVersion>
<groupId>clj-concordion</groupId>
<artifactId>clj-concordion</artifactId>
...
<build>
<sourceDirectory>src</sourceDirectory>
...
and $ head -n1 ~/Projects/clj-concordion/src/clj_concordion/core.clj
(ns clj-concordion.core
(Could there be a problem with using src/ instead of src/main/clojure/?)#2019-06-0716:05mssis there a function that pretty prints a deps.edn dependency tree a la lein deps :tree? are people just using maven to accomplish that?#2019-06-0716:06zaneYou mean like clojure -Stree?#2019-06-0716:06mssexactly what I was looking for! thanks for the help#2019-06-0716:07zaneYou're welcome!#2019-06-0717:34NolanI’ve got a java class on the classpath that looks like this:
public final class com.whatever.Main {
public static void main(java.lang.String[]);
}
Is there any way to use :main-opts to invoke com.whatever.Main.main()? I tried:
:aliases
{:some-alias
{:main-opts
["-m" "com.whatever.Main"]}}
but that looks for com/whatever/Main__init.class. I realize I’m probably stretching how aliases are meant to be used, but it would be mega convenient to be able to run this using the cli.#2019-06-0717:48zaneYou could create a Clojure namespace with a -main that just delegates to that Java method, maybe? 🤔#2019-06-0717:51Nolanyeah, that’s probably the appropriate way to go about it.#2019-06-0717:54alexmillerclj is a Clojure launcher, not a general purpose Java launcher#2019-06-0717:57NolanI subsequently realized how silly it was to be trying to use clj before there was a clojure component. Everything fell into place once I realized that the whole problem went away as soon as I started writing what I initially set out to write.#2019-06-0822:47kommenran into a an unexpected situation#2019-06-0822:48kommenthis is ok:#2019-06-0822:48kommen$ clj -Sdeps '{:deps {org.clojure/clojurescript {:git/url "" :tag "r1.10.520" :sha "0c353f1947089cb8b1f010b4294b94ac109d4ef6"} org.clojure/tools.reader {:mvn/version "1.2.2"}}}'
Checking out: at 0c353f1947089cb8b1f010b4294b94ac109d4ef6
Clojure 1.10.0
user=> (require 'clojure.tools.reader)
nil
user=> (doc clojure.tools.reader/read+string)
nil
user=> #2019-06-0822:48kommenbut this should not work, as tools.reader 1.2.2 has no read+string:#2019-06-0822:49kommen$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.520"} org.clojure/tools.reader {:mvn/version "1.2.2"}}}'
Clojure 1.10.0
user=> (require 'clojure.tools.reader)
nil
user=> (doc clojure.tools.reader/read+string)
-------------------------
clojure.tools.reader/read+string
([] [reader & args])
Like read, and taking the same args. reader must be a SourceLoggingPushbackReader.
Returns a vector containing the object read and the (whitespace-trimmed) string read.
nil
user=>
#2019-06-0822:49kommenturns out clojurescript is an uberjar which contains tools.reader 1.3.0, which takes precedence over the specified version in deps.edn#2019-06-0822:50kommenis that to be expected?#2019-06-0823:12alexmillerYes, I think so#2019-06-0823:13alexmillerThey vendor a couple things iirc, but asking in #cljs-dev would be better#2019-06-0906:45dominicmI think that clojurescript vendors an AOT version of t.r, which takes priority.#2019-06-1009:07martinklepschI've found another potential bug/issue, let me know if it makes sense to file an issue.
With the code below we'll end up with two versions of Clojure on the classpath:
(tdeps/resolve-deps {:deps {'org.clojure/clojure {:mvn/version "1.9.0"}}
:mvn/repos default-repos}
{:extra-deps {"org.clojure/clojure" {:local/root "/Users/martinklepsch/.m2/repository/org/clojure/clojure/1.10.0-beta2/clojure-1.10.0-beta2.jar"}}})
It can be easily fixed by changing the "org.clojure/clojure" to be a symbol but I would expect tools deps to handle that(?)#2019-06-1012:46alexmilleryou can file a ticket for that#2019-06-1014:42dominicmFeels like strings in that position should blow up#2019-06-1014:44dominicmHowever, strings would make a pretty neat reference to the canonical upstream for git repos, even when you're using a separate branch#2019-06-1015:09alexmillerit should be an error, libs are symbols#2019-06-1214:12unbalanceddoes anybody happen to know if there's a lein deps :tree equivalent for the command line tool?#2019-06-1214:15bronsaclj -Stree#2019-06-1214:24unbalancedyou're awesome, thanks#2019-06-1214:25unbalanceddoes this also show conflicts? My datomic-pro seems to be cranky and I'm trying to track down why#2019-06-1214:26alexmillerit doesn't - you may want to post the message in #datomic#2019-06-1214:26alexmillerthere are some common cases and someone might just know hte answer#2019-06-1214:27unbalancedaha. Thank ya sir#2019-06-1215:56unbalancedalright question... in deps.edn is there a way to specify username/password for servers? i.e., without doing this in ~/.m2/settings.xml:
<servers>
<server>
<id>my.datomic.com</id>
<username>username</username>
<password>password</password>
</server>
</servers>
#2019-06-1216:00unbalancedor perhaps another way to say this is, if I have :mvn/repos {"" {:url ""}} in my deps.edn and the ☝️ snippet in ~/.m2/settings.xml, and yet I'm still getting a 401 on my download for com.datomic/datomic-pro {:mvn/version "0.9.5786"} ... is there another way to accomplish the same thing or am I just doing it wrong?#2019-06-1216:01alexmillerthat is the right way to go about it, so something must not be lined up correctly#2019-06-1216:06unbalancedokay, I'm stupid#2019-06-1216:07unbalancedI didn't realize in the examples that was just a snippet of settings.xml#2019-06-1216:08unbalancedneed the full thing... working now#2019-06-1216:10alexmillercool, it's a balance on how much to put without it being overwhelming in those docs#2019-06-1216:21dominicmI saw something really neat recently where you could expand an example into the fully working program, this would be a good use case for that#2019-06-1217:26sveriYea, or a link to a full code example on github.#2019-06-1216:14unbalancedindeed#2019-06-1319:53jmvis it possible to set the location of downloaded artifacts? the CI we use treats each step in the build pipeline as a clean slate and so it re-downloads all of the dependencies for each step.#2019-06-1402:31alexmiller@jmv305 yes, set the :mvn/local-repo attribute at the root of deps.edn (or in -Sdeps on command line) - takes a string that is a path to the root#2019-06-1414:51jmvthank you alex, that worked!#2019-06-1402:32alexmillerclj -Sdeps '{:mvn/local-repo "/right/here"}'#2019-06-1408:14borkdudeI’m getting an error when I put clojure 1.10.1 in my extra-deps. I wanted the dependency to be there in my maven repo during test time, since I’m analyzing the source code.
;; org.clojure/clojure {:mvn/version "1.10.1"}
#2019-06-1408:15borkdudeThe error being:
Running tests in #{"test"}
Syntax error (ClassNotFoundException) compiling at (clj_kondo/core_test.clj:1:1).
clojure.core$with_meta__5405
#2019-06-1408:16borkdudeI didn’t want to declare that clojure version at my top level, since I don’t want to force that version on other people#2019-06-1408:17borkdudetrying a repro#2019-06-1408:22borkdudeI can force the download in another way, never mind#2019-06-1409:43holyjakDoes anyone know how to publish a deps.edn-based library to Clojars? Its docs do not mention it https://github.com/clojars/clojars-web/wiki/Pushing#2019-06-1409:51delaguardoI'm using maven for that. clj -Spom and then mvn deploy.#2019-06-1409:52delaguardoSo you probably need to read through https://github.com/clojars/clojars-web/wiki/Pushing#maven#2019-06-1416:01seancorfieldYup that's how I do it too. With depstar to create the JAR. #2019-06-1416:03seancorfieldThe pom generated by clj is minimal -- you'll need to add a bunch of stuff and edit things. See the pom.xml file in the depstar repo. Without all the extra stuff Clojars won't know how to refer back to your source repo and you won't be able to import your library on http://cljdoc.org either. #2019-06-1416:03seancorfieldnext.jdbc is also published this way. #2019-06-1416:09borkdudethis is good to know. I currently still mirror my deps in a project.clj and then call lein deploy#2019-06-1416:29jmvdo you update the version in the pom by hand?#2019-06-1416:31borkdudegood question. I wonder where the version comes from since that’s not present in deps.edn?#2019-06-1416:36jmvhttps://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/gen/pom.clj#L76#2019-06-1417:28seancorfieldYes, I update the pom.xml <version> tag manually. But the the SCM <tag> is updated by my deploy shell script when I make a release.#2019-06-1417:28seancorfieldSo my process the first time is clj -Spom and add all the SCM stuff etc into the skeletal pom.xml (by copying from another project and editing all the URLs etc).#2019-06-1417:31seancorfieldSubsequent times: do the edits to prepare for the release, push to GH, create a new release on GH with release notes, pull (to ensure I have the latest master locally), then clj -Spom && clj -A:jar project.jar && git status (to verify the pom.xml didn’t change again). Then I usually re-run the test suite at this point. Then deploy project, which updates the pom.xml from the HEAD SHA, commits & pushes it, then uses mvn deploy to push it up to http://clojars.org#2019-06-1417:32seancorfieldThis is how I update the pom.xml in my deploy script sha=`git rev-parse HEAD`
version=`fgrep '<version>' pom.xml | head -1 | sed 's;.*<version>\(.*\)</version>.*;\1;'`
echo "Setting tag to ${sha} for version ${version}..."
cp pom.xml /tmp/${1}.pom.xml
sed "s;<tag>[0-9a-f]*</tag>;<tag>${sha}</tag>;" < /tmp/${1}.pom.xml > pom.xml
#2019-06-1417:32seancorfieldThis is the deploy line: mvn deploy:deploy-file -Dfile=${1}.jar -DpomFile=pom.xml -DrepositoryId=clojars -Durl=
#2019-06-1417:40jmvthanks sean, this is helpful to see#2019-06-1417:43seancorfieldThe automated commit looks like this https://github.com/seancorfield/next-jdbc/commit/43f74637996ffeaef51bb1965374c784c9ec5b28 but the actual release — whatever was tagged by GH — is what Clojars points to, e.g., https://github.com/seancorfield/next-jdbc/commit/9280b9ab4753a62e14b564df0797c124af14e4ad (which is one before that).#2019-06-1417:44seancorfieldHowever, it’s the <tag> in the pom.xml that drives http://cljdoc.org when it fetches the project from GH to analyze — and that means it gets the deployed version (which is one commit ahead of the release). Just something to bear in mind.#2019-06-1417:45jmvright that makes sense#2019-06-1509:06holyjakThanks!#2019-06-2608:43dominicmAn example that doesn't use mvn: https://github.com/juxt/pack.alpha#uploading-to-clojars-or-maven#2019-06-1511:20sogaiuis anyone else using m4 or something similar to generate their deps.edn files?#2019-06-1515:27gerred👋 if I'm working on a dependency from a local source, is there any way to re-load that dependency from the repl?#2019-06-1518:02orestis@gerred if you have a local source, can’t you just reevaluate the code in the REPL?#2019-06-1716:28gerredlocal source, but different repo. for example, working on an upstream lib but want to see the effects on whatever my downstream project is. 🙂#2019-06-1716:28gerredotherwise, totally#2019-06-1716:28gerred(say, adding a feature to someone's upstream before I PR it)#2019-06-1717:28orestisShouldn’t matter which repo the code is from, as long as your editor picks up the correct namespace (most do)#2019-06-1610:45borkdude@gerred yeah, you can do (require '[my.lib] :reload)#2019-06-1611:11minhnnhow to auto reload new lib into repl when I add It to my deps.edn, It depend on IDE or I must do itt manualy with deps/resolve-deps and deps/make-classpath#2019-06-1611:22borkdude@minhnhat10bk there is add-lib in tools.deps, google for it#2019-06-1611:43minhnnI have check but It seem like no add-lib in clojure.tools.deps.alpha#2019-06-1611:46sogaiudid you check the add-lib branch? https://github.com/clojure/tools.deps.alpha/tree/add-lib#2019-06-1611:46sogaiuhttp://insideclojure.org/2018/05/04/add-lib/#2019-06-1716:21TravisHey guys, I might be missing it in the docs but is there away with tools-deps to include java src in the project ?#2019-06-1716:52ghaditools.deps doesn't compile Java @camechis#2019-06-1716:54Travisthanks, thats what I figured#2019-06-1717:12souenzzoclj idea is awesome. it should be available in every language.#2019-06-1812:41unbalancedIs there a way to download the dependencies without running the app? (use case, building Docker image)#2019-06-1812:44unbalancedclj -i?#2019-06-1812:58unbalancedahh looks like clj -version will do the trick#2019-06-1812:59kirill.salykinclj -Stree?#2019-06-1813:01unbalancedyeah that's better ... clj -version isn't a thing#2019-06-1813:01unbalancedI was doing clj -e "1" but that felt stupid#2019-06-1813:26ghadiclojure -Spath > /dev/null#2019-06-1912:01denik@alexmiller just read your post on add-lib (http://insideclojure.org/2018/05/04/add-lib/). I'm working on a project where this would be very useful. What's the status?#2019-06-1912:12vlaaadI heard some rumors Rich is thinking whether it should be in clojure core instead of tools-deps 😱#2019-06-1912:14vlaaadI wish it is more official then a separate branch because it removes SO MUCH friction#2019-06-1912:36alexmillerWe’ll get back to it at some point. Some part of it may migrate into core, hopefully that would be for 1.11#2019-06-2001:36rgmCan I pull in new code from a :local/root dependency without restarting the repl? I’m trying to extract a lib and want to have my test runner on watch mode in the main project.#2019-06-2001:37rgmer, I guess now that I think about it that’s not quite the repl. I guess I mean is there something related to add-lib that I could run from the repl or a test namespace that can reload a known :local/root dep#2019-06-2003:29seancorfield@rgm Is that code you already have on the classpath, and it just got updated? Or is this a genuine addition of new code/dependencies?#2019-06-2003:56rgmIt’s already on the class path. The plot twist is that there’s a fair bit of cljc and cljs.#2019-06-2003:58rgmLet’s imagine I have editor buffers open for both the main proj, and namespaces in the lib I’m extracting, nrepl’d or prepl’d in on the editor, and if I can avoid restarting the REPL jvm to get new lib code that’d be amazing and happy.#2019-06-2004:00seancorfieldOn the JVM, just eval code as you change it. No need for any reloading there. On the browser (cljs) side, that's what Figwheel/Shadow's hot reloading is all about. Or am I missing something @rgm?#2019-06-2004:01rgmOh, you’re probably not. The idea that it might just work had genuinely not occurred to me.#2019-06-2004:01seancorfield(I keep a JVM REPL up and running for days, sometimes weeks, and never use a watcher or reloader process)#2019-06-2004:01rgmI didn’t imagine that figwheel would observe outside the project directory.#2019-06-2004:03seancorfieldHmm, if you're editing code outside the project as well, I don't know how Figwheel et al would cope. I was just reading about some editor/REPL setup tho' where you can have both JVM and browser REPLs running and when you eval clj it goes to the JVM, cljs goes to the browser, and cljc goes to both.#2019-06-2004:04seancorfieldSo with that workflow, you'd still follow a good REPL-Driven Development workflow of edit-eval, edit-eval, without even needing to save your files.#2019-06-2004:04seancorfield(I work for fairly long times across multiple files without saving sometimes)#2019-06-2004:06rgmHuh, yeah. Maybe if I evaluate the editor buffer via prepl that could work.#2019-06-2006:48OlicalIf you're still on Conjure that's <localleader>eb :)#2019-06-2004:06seancorfieldI haven't touched cljs for over four years but I've made a public pledge to go back and spend this weekend getting familiar with modern tooling -- including Figwheel and Shadow -- although I don't know what editor I'll use for this learning project. I've moved away from nREPL (to "just" a Socket REPL or in-process REPL/REBL) but I think cljs will force me to use nREPL?#2019-06-2004:48rgmI’m having a pretty fun time with prepl and conjure (I’m a vimmer) … my experience in vim has been than nrepl + piggieback + fireplace.vim is still pretty rough. Colleagues using emacs/cider laugh at me and my struggles.#2019-06-2005:15seancorfieldI've gone through a bunch of setups over the last nine years. I think I started with TextMate and a plain REPL...#2019-06-2009:14alexmillerMe too :)#2019-06-2005:16seancorfield...tried a bunch of editor+repl combos, settled on Eclipse/CCW for a while, then moved (back) to Emacs -- I used it a lot 20-30 years ago, then switched to Atom/ProtoREPL and more recently Atom/Chlorine, which means I can get away from nREPL.#2019-06-2005:17seancorfieldAnd, lately, I've been very enamored with REBL as part of my workflow so my normal work setup is clj/`deps.edn`, Atom/Chlorine, REBL with a Socket REPL.#2019-06-2005:17seancorfieldBut I've always had a soft spot for vi so I've been keeping an eye on Liquid, which is basically an in-process vi-like editing experience with built-in evaluation (and no REPL needed).#2019-06-2005:19seancorfieldI have Liquid + REBL (started from clj/`deps.edn`) working now and will try to use that quite a bit going forward (but it does lack some parinfer/paredit/global-find'n'replace niceties).#2019-06-2005:19seancorfieldI don't know how any of that works for cljs stuff tho'. Part of this weekend's exercise will be exploring tooling and figuring all that out.#2019-06-2005:20felipebarrosYou should totally be a streamer @seancorfield... I would love to watch you experimenting with that stuff and analyze your workflow.#2019-06-2005:22seancorfield@anantpaatra Yeah, I was very interesting in the #livestream channel when it started ages ago but the problem is I mostly work on a large (86K+ line) proprietary code base and couldn't stream that.#2019-06-2005:22seancorfieldI am considering doing more screencasts (or even streaming) my OSS work tho'.#2019-06-2005:23seancorfield@anantpaatra did you see the 9 minute screencast I posted on YouTube of me working with Atom/Chlorine/REBL?#2019-06-2005:39felipebarrosIt was definitely interesting to watch that as it is a completely different workflow from everything I've seen so far. But the pace is like trying to appreciate a super-bike when it passes in front of you at 350km/h. You can hear it is loud, you can see it is expensive, it's probably beautiful and you quickly think "damn, I would love to ride that" but its gone before you even realize and you're left with your regular pedestrian day ahead of you. haha#2019-06-2005:42felipebarrosI understand it was made for experienced developers though, but I would really like that clj/deps.edn details being slowly crunched.#2019-06-2005:42felipebarrosAnd after I get that part, the REBL.#2019-06-2005:50seancorfieldOK, good feedback. I didn't realize it came across like that. I thought it was slow and gentle. I'll try harder next time.#2019-06-2005:23felipebarrosFor example, watching someone going from zero to a complex project by using clj/`deps.edn` would be awesome. I truly need something like that.#2019-06-2005:24felipebarrosI haven't! Going to check it now.#2019-06-2005:24didibus+1, not to mention you've got a great voice for it as well!#2019-06-2005:24seancorfieldYou'll be telling me next that I have an accent... 🙂#2019-06-2005:24didibus😛#2019-06-2005:26seancorfield(I'm from the South of England. I don't have an accent. Everybody else does! 🙂 )#2019-06-2005:28seancorfieldBTW, I'm going to be on DEFN Sunday morning (Pacific) and I believe they're livestreaming that. But it's just chat, not coding.#2019-06-2005:29didibusOh, cool. I'll be sure to tune in#2019-06-2005:30seancorfieldWhen I fixed an issue on core.memoize last week, I realized that it would have been a great opportunity to stream or screencast. I used REBL and tap> specifically to hone in on the bug and fix it, all live-eval'ing in the editor.#2019-06-2005:31seancorfield(and we're a bit off-topic... -> #off-topic if we want to continue this!)#2019-06-2007:44sogaiuwhere are our black boxen when we need them?#2019-06-2009:38minikomihttps://twitter.com/ryanbigg/status/1139053250244562944
➜ ~ clj -e "(clojure-version)"
"1.10.0"
#2019-06-2009:38minikomiis there a better way? 🙂#2019-06-2009:41alexmillerNot currently. There’s a ticket for it#2019-06-2015:32ghadido you show the version of clj or the default version of clojure? or the requested version of clojure in the current context?#2019-06-2019:33orestisEverything? If it’s reasonably structured it can be greppable for tools to extract more. #2019-06-2019:34orestisI think Elixir’s mix and also npm (to which clj is more alike) show the version of the tool and the version of the language (and also the host runtime version). I’ll verify at a computer tomorrow. #2019-06-2019:35ghadiall i mean is the version of the language is context dependent#2019-06-2019:35ghadi(but the version of tool + jdk aren't)#2019-06-2019:37orestisYeah for me the version of clj is more important - right now in my ansible scripts I’m looking at some some specific file to check a number. #2019-06-2019:37orestisWould prefer to invoke the tool and get an easily comparable version string without starting the JVM#2019-06-2019:38orestis(Not a big deal currently)#2019-06-2021:02alexmillerthese questions are tbd, imo. I'm not sure that the subtle difference between the clojure version used when building classpaths vs the clojure version being used in your project is obvious to most people. I don't think it's even clear to people that these are independent things.#2019-06-2021:18vlaaadI just want to share how much I love tools-deps. It's so simple in what it does, you really know what's going on when you use it, and it's so easy to just specify git dep and immediately try something: no need to publish for library authors, no need to setup a project just to try stuff out for library users... I wish we had build tools that are that simple!#2019-06-2021:18alexmillerthanks!#2019-06-2110:19stathissiderisdoes :jvm-opts work only within aliases or at the top-level too?#2019-06-2110:20kszaboonly aliases, it’s a known issue#2019-06-2110:20stathissiderisok, thanks! (glad to hear it’s a bug and not a feature)#2019-06-2117:45misha1) can I add a dependency to library so that it can be seen from library (e.g. when run as alias), but is invisible to client library? I assume deps in :aliases behave like that. and if so:
2) is there a way to depend on an alias of such a library?
Trying to have specs in my lib on one hand (for lib internal tests, and for lib's clients if they need them),
and as a lib client to be able to exclude them from dependency (in case I use another spec version, etc.)#2019-06-2117:47hiredmanno#2019-06-2117:48mishano for both 1 and 2?#2019-06-2117:48hiredmanyes#2019-06-2117:49mishawhat is the preferred way to go about it? separate repo with specs?#2019-06-2117:49mishado :aliases extra deps appear in client's class path?#2019-06-2117:50mishaif no, that makes extra repo option not that bad#2019-06-2117:50hiredmanI maybe have misunderstood 1#2019-06-2117:51mishaI want to have specs for my lib. And want you to be able to depend on my lib excluding those specs (and clojure.spec)
both at the same time#2019-06-2117:51seancorfield:aliases are not transitive across dependencies, if that's what you're asking @misha?#2019-06-2117:52hiredmanso 1, the dep on clojure.spec is going to be there, clojure pulls it in#2019-06-2117:53mishaso:
specs-repo +
lib-repo + specs-repo-as-alias-extra-dep for tests
is an ok solution#2019-06-2117:53hiredmanor just put your specs in a namespace and don't load that namespace except from tests#2019-06-2117:54mishaoh, right. then: can I not depend on clojure in my lib repo? )#2019-06-2117:54seancorfieldIn separate repos, yes, I guess that could work -- but I don't know why anyone would bother?#2019-06-2117:54seancorfieldIf your specs are part of yours tests (only) just put them in the test folder that would be added by your :test alias. Done.#2019-06-2117:55mishatrying to avoid any transitive deps you'll get by using my lib#2019-06-2117:55seancorfieldWhat transitive deps would those be? Clients of your lib need whatever your src depends on. They don't need what your test code depends on.#2019-06-2117:56mishaSean, those specs might be useful for you as a client. But if you dont care, I dont want you to get clojure.spec through my lib#2019-06-2117:56hiredmanspec comes with clojure these days, but is technically a separate artifact so you can use a newer version if you want#2019-06-2117:56seancorfieldYou can't avoid clojure.spec.#2019-06-2117:56seancorfieldClojure itself pulls in clojure.spec and core.specs. The client is always going to get those.#2019-06-2117:57mishaI need specs for tests only, but I know those might be useful for clients as well. so core does not depend on those for functionality.#2019-06-2117:57seancorfieldYou can avoid test.check (by making it a :test-only :extra-deps).#2019-06-2117:57seancorfield@misha I think you're overthinking this. If you want clients to have access to your specs as an option, put them in src. If you don't want clients to have access to them, put them in test.#2019-06-2117:58mishaok. do I have to depend on clojure itself to distribute a lib? if yes, is it better to depend on as old version as possible?#2019-06-2117:58seancorfieldThe Clojure dependency will be overridden by the client.#2019-06-2118:00mishaI am absolutely overthinking it, but on the other hand I had bad experience with different versions of direct and transitive deps before. trying to not mess things up for anyone.#2019-06-2118:12seancorfieldYeah, I hear that... If you have test-only dependencies, put them under an alias as :extra-deps. If you have source dependencies, those are going to end up as transitive dependency for your clients.#2019-06-2118:29mishaYou are right, clojure includes spec, I have to include clojure, so it boils down only to "make specs public or not". thanks, guys.#2019-06-2118:30seancorfieldAs an example, both clojure.java.jdbc and next.jdbc have all their specs in a dedicated namespace that clients can require if they wish but otherwise they are not activated.#2019-06-2118:32seancorfieldFor clojure.java.jdbc, I primarily made them optional like that so it could continue to run on older Clojure versions (that didn't support Spec). For next.jdbc, I added them after the fact as an option for folks who want better error messages (since next.jdbc sort of follows Clojure core's principle of "do the right thing as fast as possible; potentially explode in arbitrary ways if asked to do the wrong thing" (i.e., NPE or ClassCast or whatever).#2019-06-2118:48mishathis is exactly how I am approaching it now. Docstrings – contract, validation – optional, in a separate ns#2019-06-2119:45andy.fingerhutFun fact. This works: clj -Sdeps "{:deps {org.clojure/clojure {:mvn/version \"1.0.0\"}}}"#2019-06-2119:47andy.fingerhutAnd if you are wondering, I tried it because it is helpful in tracking down when some functions were added to Clojure that do not (yet) have :added metadata, but should soon.#2019-06-2119:50seancorfieldI didn't try going back any further than 1.2.1 https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L4-L15 but that's good to know @andy.fingerhut#2019-06-2119:53seancorfieldInteresting little glitch in 1.0.0: (! 563)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.0.0"}}}'
Clojure 1.0.0-
user=> (clojure-version)
"1.0.0-"
user=> *clojure-version*
{:major 1, :minor 0, :incremental 0, :qualifier ""}
user=> ^D
Fri Jun 21 12:53:28
(sean)-(jobs:0)-(~/clojure)
(! 564)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.1.0"}}}'
Clojure 1.1.0
user=> (clojure-version)
"1.1.0"
user=> *clojure-version*
{:major 1, :minor 1, :incremental 0, :qualifier ""}
user=>
#2019-06-2119:55andy.fingerhutI'll file a ticket for a 1.0.1 release 🙂#2019-06-2120:04alexmillerRight :)#2019-06-2222:21seancorfieldA screencast about the CLI, deps.edn, and aliases https://youtu.be/CWjUccpFvrg (this time in 1080p instead of 360p!).#2019-06-2306:53felipebarrosThank you so much for that! The pace is elegant, the scope is healthy and the perspective is intimate and clarifying. Excellent!#2019-06-2305:29steveb8nQ: I can’t get :override-deps to work with :local/root deps. e.g. I have a default :git/sha dep and then for local dev I want to override that with a :local/root dep. Has anyone else made this work?#2019-06-2305:34steveb8nstarting a separate thread to stop annoying others#2019-06-2305:35seancorfieldThis sounds like it should work, but I’d have to see the relevant parts of the deps.edn file I think…#2019-06-2305:35steveb8nmy specific problem is that the the :git/sha dep is not reachable in local dev so I’m trying to override it. however, with the override, it still tries to pull the :git/sha dep and that breaks#2019-06-2305:36steveb8nI agree. it should work. however it appears that with an :override-deps, the overridden dep must still be accessible#2019-06-2305:37steveb8nlet me see if I can make a simple test case deps.edn to demo it#2019-06-2305:39seancorfieldI tried from the command line with an alias to provide override-deps and it seems to work…#2019-06-2305:41seancorfieldYup, deleted my ~/.gitlibs folder and the :dev alias that overrides the dep with :local/root works fine. No attempt is made to clone the git repo.#2019-06-2305:44steveb8nok, here’s a repro (I think)#2019-06-2305:44steveb8n{:deps {reifyhealth/specmonstah {:mvn/version "2.0.0-alpha-1"}}
:aliases {:local {:override-deps {reifyhealth/specmonstah {:git/url ""
:sha "8e47ffc60d3177cea90f2f657e73c48971d8a1e4"}}}}}
#2019-06-2305:45steveb8nyuck formatting#2019-06-2305:45steveb8nthen clojure -R:local -Spath | tr ‘:’ ‘\n’ | sort#2019-06-2305:46steveb8nshows the alpha version, not my fork#2019-06-2305:46steveb8nnot quite the same as what I described but I think it might show how I am using the CLI opt wrong#2019-06-2305:50steveb8nhmm, I wonder if my version of tools.deps is too old and this is a fixed bug. I’ll check that now#2019-06-2305:51seancorfield(! 640)-> clj -Spath | tr ':' '\n' | sort| fgrep specmon
/Users/sean/.m2/repository/reifyhealth/specmonstah/2.0.0-alpha-1/specmonstah-2.0.0-alpha-1.jar
Sat Jun 22 22:50:18
(sean)-(jobs:0)-(~/clojure/steve)
(! 641)-> clj -A:local -Spath | tr ':' '\n' | sort | fgrep specmon
/Users/sean/.gitlibs/libs/reifyhealth/specmonstah/8e47ffc60d3177cea90f2f657e73c48971d8a1e4/src
Sat Jun 22 22:50:24
(sean)-(jobs:0)-(~/clojure/steve)
(! 642)-> cat deps.edn
{:deps {reifyhealth/specmonstah {:mvn/version "2.0.0-alpha-1"}}
:aliases {:local {:override-deps {reifyhealth/specmonstah {:git/url ""
:sha "8e47ffc60d3177cea90f2f657e73c48971d8a1e4"}}}}}
works for me#2019-06-2305:51seancorfield-R:local behaves the same, I checked.#2019-06-2305:51seancorfield(! 644)-> clj -Sdescribe
{:version "1.10.1.447"
#2019-06-2305:53steveb8nI’m 1.9.0.391#2019-06-2305:53steveb8ndoesn’t seem likely it’s a bug but I guess I should rule that out#2019-06-2305:54seancorfieldYeah, I think this is a bug that was fixed (ages ago).#2019-06-2305:54steveb8nyeah. I just ran the same commands you pasted in and local does not change the result#2019-06-2305:55steveb8nI’ll trying upgrading#2019-06-2305:55steveb8nnot sure how 😉#2019-06-2305:55steveb8nI’ll go find the docs for upgrading#2019-06-2305:55seancorfieldMac or Linux?#2019-06-2305:56steveb8nmac. I think it’s “brew upgrade clojure”#2019-06-2305:56steveb8ntrying that now#2019-06-2305:57seancorfieldYup, that’s the right command.#2019-06-2305:57steveb8nwaiting on Aussie broadband 🙂#2019-06-2305:57seancorfieldSorry. Mine’s about 150M down and 7.5M up.#2019-06-2305:58steveb8nmines 94m down which is really good for Aussie. I can’t complain. I suspect the latency hurts despite the good bandwidth#2019-06-2305:59seancorfieldYah, we have global monitors for our work stuff and the latency from Asia/Australia is pretty bad.#2019-06-2305:59seancorfield(our main data center is US East Coast and our primary AWS region is also US East Coast)#2019-06-2306:02steveb8nok. that fixed it in the repro case! brilliant#2019-06-2306:03steveb8nlet me check my original#2019-06-2306:03steveb8nyep, fixed there too#2019-06-2306:03steveb8nthanks for being a sounding board/tester 🙂#2019-06-2306:03steveb8nI’ll try to pay it forward#2019-06-2306:03seancorfield1.9 is super old. So many changes, fixes, enhancements since then!#2019-06-2306:04seancorfield(but then I’m the guy who always runs Clojure alpha/beta builds in production! LOL!)#2019-06-2306:04steveb8nyeah. I’m using 1.10 for my project, just not for the CLI that kicks it all off#2019-06-2306:04steveb8nnow I am 🙂#2019-06-2306:05steveb8nhave a good day/night. and thanks again for the videos, I’m gonna go watch the rest of them now#2019-06-2306:14seancorfieldG’nite! It’s 11:15pm here and definitely bedtime.#2019-06-2305:30steveb8nbtw: @seancorfield I just watched your video on deps.edn. really useful. thanks.#2019-06-2305:33seancorfieldGlad it was helpful!#2019-06-2305:34seancorfieldCan you share your deps for the git/url and local/root deps in the alias?#2019-06-2319:34didibusIs there a way to specify jvm options but not for an alias?#2019-06-2319:35didibusAnd not at the command line#2019-06-2319:35didibusSo have them specified in deps.edn for when I run main#2019-06-2320:01seancorfieldRight now they only work inside an alias @didibus — it’s a known issue/limitation. Sounds like the team may change that at some point…#2019-06-2320:02didibusI see, thx !#2019-06-2321:41andy.fingerhutDoes anyone happen to know when you make a text file beginning with #! /usr/bin/env clojure on a Unix/Linux system, once the JVM begins and the Clojure reader starts reading the file, is the Clojure reader also given that first line to read as well?#2019-06-2321:41andy.fingerhutOr is there some code somewhere that somehow skips over it, and thus the Clojure reader never sees it?#2019-06-2321:42andy.fingerhutAnd if so, do you know where that "skipping" code is?#2019-06-2321:49alexmillerIn LispReader#2019-06-2321:50alexmillerIt skips lines that start with #!#2019-06-2321:50alexmillerOr rather, those act as single line comments#2019-06-2321:51andy.fingerhutthx. Someone was asking how this script worked, and I think I knew all of the magic, except that part. Now my magical knowledge is complete! https://gist.github.com/ericnormand/6bb4562c4bc578ef223182e3bb1e72c5#2019-06-2321:54alexmillerIt was certainly a future looking escape in the reader :)#2019-06-2321:57andy.fingerhutAnd I have a vague feeling that I have asked this and/or discovered it before, and was also then surprised that it has been around since 2009#2019-06-2321:59andy.fingerhutDo you think it belongs in the Reading Clojure Characters guide, or better to keep it obscure? 🙂#2019-06-2322:09alexmillerWell I don’t think it belongs there as it’s not really part of the Clojure language#2019-06-2322:10alexmillerReally more syntax intentionally ignored by Clojure #2019-06-2322:10alexmillerCould be on the reader page#2019-06-2323:30andy.fingerhutI haven't followed the myriad ways people have used deps and clj/clojure, but does the approach on that page: https://gist.github.com/ericnormand/6bb4562c4bc578ef223182e3bb1e72c5 pretty much a good way to go, if you want to embed the JVM and/or clj/clojure options within the same file as the Clojure code? As in, have people found other ways to combine those options together into the same file as the Clojure code, so that those options do not end up in a separate file/script somewhere?#2019-06-2412:08futuroI love the use of #_ to hide the shell opts from the Clojure reader. #2019-06-2416:18andy.fingerhutIt does restrict you a little bit, in that those first 16 lines must be valid shell syntax, as well as legal Clojure syntax, but yeah, I am pretty sure it is not the first language where someone has found a handful of dual-language lines of code.#2019-06-2417:33futuroHmm, that’s a good point. #2019-06-2401:33sogaiuas another data point: https://gitlab.com/eval/ohmyclj/#2019-06-2423:22rgmSo I’m running macOS 10.15 betas and seem to have gotten bit by https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-91 … it manifests as a failure to clone a private lib from a :git/url, whether using an id_rsa created by under macOS 10.15 or earlier.#2019-06-2423:23rgmis the new jira taking new commenters yet? Wanted to update TDEPS-91#2019-06-2423:33andy.fingerhutThe new JIRA does allow adding comments, if you have a free account created, and if that account's permissions have been bumped up by a Clojure JIRA admin.#2019-06-2423:34rgmoh, cool… is that someone less busy than Alex?#2019-06-2423:34andy.fingerhutAt least, the older Clojure JIRA instance required someone to bump up your permissions. My permissions allow me to do so with the new JIRA, but I believe Alex bumped up most/all people who have created or commented on issues in the old JIRA, in the new one.#2019-06-2423:35andy.fingerhutI had permission to do that on the old JIRA system, but not sure whether I do on the new one or not. Let me take a look. Otherwise, I suspect it would not be someone less busy than Alex.#2019-06-2423:36rgmno rush; I have enough of a handle on what’s going on in ~/.gitlibs that I’ve just manually set it up so clj et al don’t complain any more, and I can do manual git pulls for a while.#2019-06-2423:38rgmjust wanted to bump this since it’s gonna bite a lot of people come 10.15 release in September.#2019-06-2423:39andy.fingerhutTo be clearer, I mean that my account on the new Clojure JIRA gives me permission to add comments on issues. I do not know how on the new JIRA to give permission to others to do so, and/or my account on the new JIRA doesn't have authorization to do so.#2019-06-2423:39rgmah, gotcha. I’ll ask Alex.#2019-06-2501:00ghadi@rgm an id_rsa key should work just fine. Are you running under an ssh agent?#2019-06-2501:01rgmI am.#2019-06-2501:01ghadiis the key inside your agent?#2019-06-2501:01ghadissh-add -l#2019-06-2501:02ghadishould verify ^#2019-06-2501:05ghadiI've edited TDEPS-91 to be problem-oriented rather than leading with a solution#2019-06-2501:14rgmyep, it is.#2019-06-2501:14rgmthe same key file works under 10.14 and fails under 10.15, same clj version#2019-06-2501:15rgmunder an agent both places.#2019-06-2501:21ghadiif you can find out what the difference is, I'd be much obliged#2019-06-2501:21ghadiis SSH_AUTH_SOCK set?#2019-06-2501:50rgmhm, good question. It appears yes.#2019-06-2501:52rgm#2019-06-2615:07credulousHi! I’m trying to convert my workflow to clj/deps.edn, and I’m having a bit of trouble with depstar, which I am trying to use to package my app.#2019-06-2615:07ghadiseancorfield/depstar @credulous?#2019-06-2615:07ghadican you describe what you expected and what you are experiencing?#2019-06-2615:08creduloushealthfinch/depstar I think?#2019-06-2615:08credulousYes I can… I added this to my ~/.clojure/deps.edn:#2019-06-2615:08credulous:depstar {:extra-deps {com.healthfinch/depstar {:git/url ""
:sha "4aa7b35189693feebc7d7e4a180b8af0326c9164"} }
:main-opts ["-m" "hf.depstar.uberjar"]
}#2019-06-2615:08ghadiswitch to the fork (I originally wrote healthfinch/depstar but now sean has the canonical repo)#2019-06-2615:08credulousOK!#2019-06-2615:09credulousI’ll try that before I describe anything more.#2019-06-2615:09credulousThanks#2019-06-2615:09ghadihis README is good, too#2019-06-2615:09ghadibetter#2019-06-2615:09credulousIn his blog post, Sean is still linking to yours#2019-06-2616:00seancorfieldI'll update that blog post when I get to my desk -- thanks for catching that!#2019-06-2617:01seancorfieldUpdated. Make take a few minutes to clear the cache!#2019-06-2615:09ghadiwhich blog post?#2019-06-2615:10credulousIt’s over a year old. I should have been paying more attention. https://corfield.org/blog/2018/04/18/all-the-paths/#2019-06-2615:13credulousYup, the fork works. Thanks @ghadi!#2019-06-2615:15credulousAnd may I also say: @seancorfield is everywhere! What a contributor.#2019-06-2615:18ghadihe is a treasure#2019-06-2615:20alexmillerMVP!#2019-06-2617:01credulousThanks! Also, I’m floundering a bit trying to include my public folder in the uberjar, using depstar. I created an alias with {:extra-paths ["public"]}, but that includes the contents of the public folder into the root of the uberjar. Any advice?#2019-06-2617:02seancorfield@credulous So you have resources/public -- would {:extra-paths ["resources"]} do what you need?#2019-06-2617:03credulousNo, in my project folder I have /resources and /public#2019-06-2617:03seancorfieldDon't do that 🙂#2019-06-2617:03credulousOK!#2019-06-2617:03seancorfieldPut public inside resources -- then your "source" paths are going to be "src" "resources"#2019-06-2617:04seancorfieldAnd in your code, specify that pages etc come from "public" rather than the root of your classpath.#2019-06-2617:05credulousSounds good. I was cribbing from a complete project on github… boodle, I think.#2019-06-2617:05seancorfield{:root "public"} in Compojure's resources call (assuming you're using that?)#2019-06-2617:07credulousI’m probably using compojure wrong too… I’m not using :root anywhere#2019-06-2617:07credulous(compojure/defroutes app
(-> (compojure/routes
(route/resources "/")
api/routes )
reload/wrap-reload))
#2019-06-2617:07credulouswhere route/resources defaults to “public” as the source directory, I think#2019-06-2810:36vlaaadCan someone explain how do I build a classpath properly?
(deps/make-classpath
(deps/resolve-deps {:deps {'re-find {:git/url ""
:sha "c27af619a9d9e12832b7446e127d68ab916ce78c"}}}
nil)
nil
nil)
produces this classpath:
- .../tools.cli-0.4.1.jar
- .../math.combinatorics-0.1.4.jar
- .../clojure-1.8.0.jar"
but there should also be .gitlibs/libs/refind/...sha/src, no?#2019-06-2811:54vlaaadApparently I had to add :paths ["src"] to map, which works, but feels a bit wrong... shouldn't it be the default? if I'll set :paths to ["src/clj"] in my ~/.clojure/deps.edn, will it break dependencies because I overridden their source path?#2019-06-2811:54vlaaadApparently I had to add :paths ["src"] to map, which works, but feels a bit wrong... shouldn't it be the default? if I'll set :paths to ["src/clj"] in my ~/.clojure/deps.edn, will it break dependencies because I overridden their source path?#2019-06-2812:54alexmiller:paths ["src"] is in the install-level deps.edn, but when you're using the library directly, you're in full control of what you include#2019-06-2812:55alexmillersetting to "src/clj" only affects your lib, not transitive deps - they set their own paths in their own deps.edn#2019-06-2812:59vlaaadthat's the thing, re-find does not set :paths explicitly, that's why I need to specify ["src"]#2019-06-2813:00vlaaadit all works until everyone's system :paths property is the same#2019-06-2813:00vlaaadsetting "src/clj" affects dependent libs too#2019-06-2813:03vlaaad(run! println
(str/split
(deps/make-classpath
(deps/resolve-deps
{:deps {'re-find {:git/url ""
:sha "c27af619a9d9e12832b7446e127d68ab916ce78c"}}
:paths ["src/clj"]}
nil)
nil
nil)
#":"))
=>
/home/vlaaad/.gitlibs/libs/re-find/re-find/c27af619a9d9e12832b7446e127d68ab916ce78c/src/clj
/home/vlaaad/.m2/repository/org/clojure/tools.cli/0.4.1/tools.cli-0.4.1.jar
/home/vlaaad/.m2/repository/org/clojure/math.combinatorics/0.1.4/math.combinatorics-0.1.4.jar
/home/vlaaad/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar
#2019-06-2813:05vlaaadre-find sources are implicitly on src, not src/clj, but it gets overridden#2019-06-2813:15alexmillersorry, I don't understand what you're saying#2019-06-2813:16alexmillerit looks like answer above is what I would expect to see#2019-06-2813:16alexmillerit looks like answer above is what I would expect to see#2019-06-2813:18vlaaadI am surprised that I have to specify :paths to get code from git dependency#2019-06-2813:19vlaaadI expect gitlibs to have default :paths ["src"]#2019-06-2813:22alexmilleroh, I see what you're saying#2019-06-2813:23alexmillerit's up to libs to specify their own paths and re-find could (probably should) set :paths ["src"], and if it did, that would be used#2019-06-2813:23alexmilleryou're seeing src/clj there due to a bug, where if no paths are used, the paths in the root deps.edn are used#2019-06-2813:26alexmillerhttps://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-96 (may be other tickets that cover this too)#2019-06-2813:31alexmillerhttps://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-52 is same thing#2019-06-2813:36vlaaadah, it's already reported, nice#2019-06-2813:39alexmilleryeah, the cause is bound up in some other issues so I've had a long mull about how to fix, but ready to move forward on it when I have some time#2019-06-2814:22vlaaadclojure.tools.deps.alpha.specs seem wrong#2019-06-2814:22vlaaad(s/def :local/root string?)
(s/def :local/coord (s/keys :req-un [:local/root] :opt-un [::path]))
#2019-06-2814:22vlaaadshouldn't it be :req instead of :req-un?#2019-06-2814:35alexmilleryes, I think you're right#2019-06-2814:35alexmillerfixed#2019-06-2821:33drewverleeI'm attempting to build an uberjar using deps + uberdeps Everything works as expected until i try to run the uberjar and it tells me it cant find clojure.main.
➜ cat deps.edn
{:aliases {
:uberjar {:extra-deps {uberdeps {:mvn/version "0.1.4"}}
:main-opts ["-m" "uberdeps.uberjar"]}}}
➜ clj -A:uberjar
[uberdeps] Packaging target/frog.jar...
[uberdeps] Packaged target/frog.jar in 2 ms
➜ cat src/core.clj
(ns frog.core
(:gen-class))
(defn -main [& args]
(println "HIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII"))
➜ java -cp target/frog.jar clojure.main -m frog.core
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
As a starting point, im not sure why the instructions from uberdeps have me look for clojure.main, if its an uberjar isn't it included?#2019-06-2821:35hiredmanif you don't have clojure in your deps#2019-06-2821:35hiredmansome of it depends on how smart the uberdeps library is#2019-06-2821:36hiredmanif you don't include a dep on clojure explicitly, you will fallback to whatever is in the system (or whatever it is called) deps.edn file that was installed when you installed clj#2019-06-2821:37hiredmanbut uberdeps might not be processing your deps.edn file using the tools.deps code, so it might not know to do that fallback#2019-06-2821:37hiredmanso if you don't get include a clojure version in your deps.edn, there wouldn't be clojure in your jar#2019-06-2821:37hiredmanyep#2019-06-2821:38hiredmanit just slurps in the edn and reads it#2019-06-2821:38hiredmana lot of tools built on top of deps.edn are super immature and very broken, and they largely duplicate each other#2019-06-2821:39hiredmanthis is the second broken uberjar building library I have diagnosed today (the other tool seemed to end up aot compiling things twice)#2019-06-2821:41hiredmanalso, if you are not aot compiling you don't need the :gen-class#2019-06-2822:08drewverleeI thought it was necessary to for the rest of the tooling to convert the clojure code to java.#2019-06-2822:16drewverleeOh, your saying that if I don't specify that, the compiling still happens, just not ahead of time?#2019-06-2822:19hiredmanclojure code is not converted to java, it is compiled to jvm bytecode#2019-06-2822:20hiredmanit is always compiled to jvm bytecode, either when the code is loaded and right before it is run, or you load the code ahead of time and save the compiled bytecode and run the compiled bytecode (aot compilation)#2019-06-2822:20hiredman:gen-class only does anything when you are aot compiling#2019-06-2822:20hiredmanand it is only needed in specific circumstances#2019-06-2822:20hiredmanyou are not aot compiling so you do not need it#2019-06-2822:31drewverleeAlmost understand what your saying, I'll need to read a bit. Thanks!#2019-06-2821:44alexmillera new version of tools.deps is out now that probably makes it much harder to do the wrong thing with the install deps.edn#2019-06-2821:45hiredmanI dunno, this code literally just slurps the deps.edn and calls read-string on it#2019-06-2821:45seancorfield@drewverlee Try using depstar instead of uberdeps -- unless there's some specific feature you need from the latter?#2019-06-2821:49drewverleeThere isn't! I did try both, I think I got a similar error. For one, It's possible I have an old version of deps. I'll have to check when I'm at my computer again. I'm always blown away by how fast everyone responds.#2019-06-2821:50seancorfieldHappy to help with any issues you have with depstar since I maintain it.#2019-06-2823:46drewverleeSo it resulted in a FileNotFoundException:
➜ cat deps.edn
{:deps {org.clojure/clojure {:mvn/version "1.10.0"}}
:aliases {:depstar
{:extra-deps
{seancorfield/depstar {:mvn/version "0.2.1"}}}}}
➜ clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
Building uber jar: MyProject.jar
➜ frog java -cp MyProject.jar clojure.main -m frog.core
Exception in thread "main" java.io.FileNotFoundException: Could not locate frog/core__init.class, frog/core.clj or frog/core.cljc on classpath.
at clojure.lang.RT.load(RT.java:466)
at clojure.lang.RT.load(RT.java:428)
at clojure.core$load$fn__6824.invoke(core.clj:6126)
at clojure.core$load.invokeStatic(core.clj:6125)
at clojure.core$load.doInvoke(core.clj:6109)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5908)
at clojure.core$load_one.invoke(core.clj:5903)
at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
at clojure.core$load_lib.invokeStatic(core.clj:5947)
at clojure.core$load_lib.doInvoke(core.clj:5928)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$load_libs.invokeStatic(core.clj:5985)
at clojure.core$load_libs.doInvoke(core.clj:5969)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$require.invokeStatic(core.clj:6007)
at clojure.main$main_opt.invokeStatic(main.clj:491)
at clojure.main$main_opt.invoke(main.clj:487)
at clojure.main$main.invokeStatic(main.clj:598)
at clojure.main$main.doInvoke(main.clj:561)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:37)
➜ clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
Building uber jar: MyProject.jar
➜ java -cp MyProject.jar clojure.main -m frog.core
Exception in thread "main" java.io.FileNotFoundException: Could not locate frog/core__init.class, frog/core.clj or frog/core.cljc on classpath.
at clojure.lang.RT.load(RT.java:466)
at clojure.lang.RT.load(RT.java:428)
at clojure.core$load$fn__6824.invoke(core.clj:6126)
at clojure.core$load.invokeStatic(core.clj:6125)
at clojure.core$load.doInvoke(core.clj:6109)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5908)
at clojure.core$load_one.invoke(core.clj:5903)
at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
at clojure.core$load_lib.invokeStatic(core.clj:5947)
at clojure.core$load_lib.doInvoke(core.clj:5928)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$load_libs.invokeStatic(core.clj:5985)
at clojure.core$load_libs.doInvoke(core.clj:5969)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$require.invokeStatic(core.clj:6007)
at clojure.main$main_opt.invokeStatic(main.clj:491)
at clojure.main$main_opt.invoke(main.clj:487)
at clojure.main$main.invokeStatic(main.clj:598)
at clojure.main$main.doInvoke(main.clj:561)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:37)
➜ cat src/core.clj
(ns frog.core)
(defn -main [& args]
(println "Hello World"))
➜ clj -Spath
src:/home/drew/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar:/home/drew/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/drew/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
I assume because as src/ is on my class path (which is default right?). Then it would pick up the core.clj file with the frog.core ns. But either it doesn't pick it up, or i'm not specifying it correctly.
It seems to be in the jar file:
➜ cat core.clj
(ns frog.core)
(defn -main [& args]
(println "Hello World"))
#2019-06-2900:02seancorfieldIs that really src/core.clj or src/frog/core.clj?#2019-06-2900:03seancorfield(it should be the latter: the file path must match the namespace segments!)#2019-06-2900:03seancorfieldA good sanity check is to see whether clj -m frog.core works before you try to build the JAR.#2019-06-2900:07drewverleeah ok. i guess i didn't realize the relationship between the files and the ns was more then convention.#2019-06-2900:22seancorfieldWhen you ask Clojure to run frog.core, how is it going to find it? The classpath is all file paths, so the ns has to be mapped to a file path to perform that search if it can't already find the compiled class in memory... so that's frog/core.clj (or frog/core.cljc) relative to each folder on the classpath, e.g., src.#2019-06-2821:45alexmillerwell, if you're not using tools.deps at all, then yes, that's prob broken#2019-06-2821:45seancorfieldSome of these packaging libs try to create a classpath which is not what you would have when running your code 😞#2019-06-2821:46alexmillerbut I would encourage whomever to file tickets on these libs so they can be better, rather than raging about brokenness#2019-06-2821:46seancorfield(I hadn't even heard of uberdeps before just now)#2019-06-2821:47hiredmanI could be wrong too, I am just guessing at the problem#2019-06-2821:51hiredmanI am just suggesting, like, be aware of that state of maturity of a lot of these libraries(tools.deps is still new, so tooling built on it is even newer), and maybe vet them a little more before using them.#2019-06-2821:52andy.fingerhutWell, vetting includes using, at least a little bit 🙂#2019-06-2908:31dominicmI'm fairly confident that pack does the right thing™. We had some early issues due to the system deps.edn (although it sounds like I'm able to remove that now! 🎉). I think pack is the oldest library in this space, and I've been incredibly careful about things like classpath collisions, which other tools have been more unconcerned about.#2019-06-2911:37joseI just installed clojure 1.10.1.455, using https://download.clojure.org/install/linux-install-1.10.1.455.sh. Running clj command starts a REPL with clojure 1.9, while if I install the previous version, 1.10.447, clj command starts a REPL with clojure 1.10.1.
I think the reason is that in the latest released jar, at clojure/tools/deps/deps.edn, clojure version is set to 1.9 and that I don't require a clojure version on my ~/.clojure/deps.edn
Is this the intended behavior?#2019-06-2912:20alexmillerOh, that is a bug#2019-06-2912:20alexmillerI know exactly why that’s happening#2019-06-2912:20alexmillerThx#2019-06-2912:38alexmiller@jlle 1.10.1.458 is now available, should fix that#2019-06-3008:38joseTested it, now clj start a REPL with 1.10.1. Thanks for fixing it!#2019-06-2914:47alexmillerFor the toolsmiths out there, I'm going to deprecate the reader/clojure-env call in tools.deps.alpha (and eventually remove it). Afaik, this was being used exclusively to obtain the list of config files used by clj. As of the latest tools.deps.alpha, 0.7.516, there is a new call reader/default-deps that will return the list of deps.edn config files. The install-level deps.edn is now embedded and this function replicates the logic from clj to find the user-level deps.edn and project-level deps.edn without shelling out (so also OS-safe). I sent a few PRs to places where I could find people doing this. @dominicm @olical @seancorfield @taylor#2019-06-2915:08OlicalWonderful, thank you very much!#2019-06-2915:05dominicmI think that I embedded my own copy of the system deps.edn a while ago. I'll make an issue to remove it and switch to whatever I'm supposed to be doing now.#2019-06-2915:08alexmillerlmk if you have any questions#2019-06-2915:11dominicmlooks like the read-deps call in pack can now be a direct call to t.d.a, I was doing something very similar with a slurp to the deps.edn file.#2019-06-2915:12dominicmquite ot, but why did you ^:const the path to the install deps.edn? As in, what made you think to do it? j/w, I've never really seen code use ^:const. Oh, is it because of optimizing performance in t.d.a as much as possible in order to minimize startup time?#2019-06-2915:20alexmillerit is literally a string constant, so it can be inlined#2019-06-2915:21alexmillerI think there is no chance of any perf difference you'd notice :)#2019-06-2918:44alexmillerNew clj 1.10.1.458 and tools.deps.alpha 0.7.516 are now available https://groups.google.com/forum/#!topic/clojure/trzlw4gCobM#2019-06-2918:46alexmiller@vlaaad this fixes the thing you pinged about yesterday#2019-06-2918:48vlaaadNice, thanks!#2019-06-3001:53seancorfieldNice round of bug fixing on t.d.a.! Inspired me to bash a bunch of bugs in clj-new and to cut new releases of depstar and clj-new 🙂#2019-06-3001:54alexmillerYep, felt good to do finally#2019-06-3001:55alexmillerI’m trying to decide if there’s anything left that’s critical before pulling out of alpha#2019-06-3001:56seancorfieldTDEPS-91 ?#2019-06-3001:58seancorfieldI'm a bit surprised that TDEPS-132 isn't fixed by some of the changes in 0.7.516? Or is that just waiting for @dominicm to see what state that ticket should be in with the latest version?#2019-06-3001:59alexmillerThe git stuff is all independent of api. Plenty of things to fix/add still, just don’t think they will affect the api#2019-06-3002:01alexmillerI haven’t looked at 132, not sure what it needs still#2019-06-3002:02alexmillerBut don’t think it affects api#2019-06-3002:04seancorfieldAh, OK. Yeah, I guess TDEPS-91 is pure implementation.#2019-06-3002:04seancorfieldWhat about add-lib at this point?#2019-06-3002:07seancorfieldWill you be updating the add-lib branch from master @alexmiller?#2019-06-3002:08seancorfield(I'm toying with the idea of doing a screencast of building a small web app from scratch but was thinking of using add-lib to show how to add dependencies while you're working live in an app... thoughts?)#2019-06-3002:09alexmilleradd-lib is additive so, wouldn’t affect alpha. Not sure yet where that will go#2019-06-3002:10alexmillerI can update add-lib if you need it, although it’s not far behind master#2019-06-3002:11seancorfieldIt would be nice to keep it current with each release "just because" 🙂#2019-06-3004:12alexmillerupdated, new sha d0b33e0d346736aa985c150145b332f97b92135e#2019-06-3005:26seancorfieldThank you!#2019-06-3005:42seancorfieldMy dot-clojure repo is updated to use that SHA for :deps.#2019-06-3005:43seancorfieldNot sure when I'll do the add-lib screencast. I'll probably try to find time this week.#2019-06-3021:23geraldodevThis "Bill of materials" from maven is not existent on tools-deps because it already compute the transitive dependencies rightly ? Is it like a in advanced package-lock.json ?#2019-06-3021:29seancorfield@geraldodev Not quite sure what you're asking there?#2019-06-3021:33seancorfieldWith Maven, each library lists only what it depends on directly -- and then Maven figures out all the transitive dependencies...#2019-06-3021:34geraldodevhttps://stackoverflow.com/questions/27100141/what-is-bill-of-materials-coming-in-spring#2019-06-3021:34geraldodevI'm trying camunda-bpm and it has a bill of materials artifact#2019-06-3021:36geraldodevon mvnrepository it advises to import into lein like this [org.camunda.bpm/camunda-bom "7.11.0" :extension "pom"]#2019-06-3021:36seancorfieldJust read that, still no idea what it is that's any different from specifying dependencies...#2019-06-3021:38seancorfielddeps.edn supports :extension#2019-06-3021:39seancorfield{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "5b283e3a35381b1f1e08100f140b760a0e690e116a15"}, :content ("[email protected]")} is that what you're looking for?#2019-06-3021:40geraldodevThe right way of using this camunda-bom#2019-06-3021:42seancorfieldI thought there was a syntax involving $ for classifier support but this seems something different (simpler).#2019-06-3021:45geraldodevwhat :extension "pom" does ?#2019-06-3021:45seancorfieldAh, here you go: https://clojure.org/reference/deps_and_cli#_dependencies -- that describes :extension (as above) and the $ syntax for classifiers.#2019-06-3021:46seancorfieldI don't know much about Maven but my understanding is that :extension "pom" tells the system to look for a .pom file instead of a .jar file -- which is what you need for BOM, right?#2019-06-3021:48geraldodevFrom what I understood this bom is a dependency that its like a package-lock.json and later the artifact is enlisted as a dependency minus its version, and maven uses this blessed list to fetch the right version#2019-06-3021:49seancorfieldI've no idea what a package-lock.json file is, BTW.#2019-06-3021:49geraldodevif you clj -Stree '{:deps {org.camunda.bpm/camunda-bom {:mvn/version "7.11.0" :extension "pom"}}}'|grep camunda camunda its not listed, so I assume it's not being seen like a dependency#2019-06-3021:51geraldodev@seancorfield thx helping me. I'll no use the BOM artifact.#2019-06-3021:52seancorfieldYeah, so it looks like you'd have to specify all the camunda deps explicitly in deps.edn if pulling in via the bom doesn't work...#2019-06-3021:54seancorfield@alexmiller can probably explain more of the differences between how lein/`boot` work and how tools-deps works around that.#2019-06-3021:56geraldodeveven there I think its not automatic because there is a https://github.com/tsachev/lein-bom plugin#2019-06-3021:57seancorfieldAh, interesting. Maven is a giant mystery to me and I try hard to avoid it as much as possible 🙂#2019-06-3022:01geraldodevme too, but I'm trying to use camunda-bpm-engine on a tools-deps project#2019-07-0113:47dharriganThis may be a daft question, but if I switch JVMs (from 12 to 8 ) on the command line, and I restart the repl, do I need to clean up anything to ensure that JVM 12 bytecode is gone? Or is it all in memory etc...?#2019-07-0113:47dharriganI'm trying to track down a really really difficult to diagnose interop issue and I'm starting to run out of options.#2019-07-0113:48ghadisame as lein#2019-07-0113:48ghadiit's all in memory unless you've AOTed something to disk#2019-07-0113:49ghadiyou should post more specifics about the interop issue @dharrigan#2019-07-0113:50dharriganI've posted the problem in #interop, no response yet.#2019-07-0113:50alexmillerbytecode is only created if you 1) AOT (and you typically tell it where to go, like target/classes) or 2) in memory when loading source. It's not cached anywhere else.#2019-07-0113:52dharriganbasically (and I don't really want to get into a crosspost/in-depth here), a one-liner in Kotlin message.toByteArray() where message = protobuf, fails miserably with clojure (.toByteArray message), with clojure complaining about illegalaccesserror. So, I even tryed to extract that code into a little kotlin module by itself and invoke kolin code from clojure (that does the same thing) but I see the same issue. It feels like the JVM isn't finding all the classes <shrug> no idea.#2019-07-0113:53alexmillerprob better to have this discussion in #interop#2019-07-0113:53dharriganI see your message 🙂#2019-07-0116:51Traviswith tools deps is there a way to add built java classes to the classpath ? For instance I have a project using GRPC which is protobuf java classes that are generated.#2019-07-0116:52ghadi:paths#2019-07-0116:53ghadi^ corresponds to classpath directly#2019-07-0116:53ghadior :extra-paths if you're in an alias#2019-07-0116:53Travisok, so just add a path to where the classes exist#2019-07-0116:53ghadiyup#2019-07-0116:54ghadito the output directory of protobuf, probably#2019-07-0116:54Travisgotcha, wasn’t sure if that worked for class files#2019-07-0116:54ghadifor $dir/foo/bar/Something.class
you'd add $dir#2019-07-0116:54Travisperfect#2019-07-0119:34dnolenhrm in order to resolve conflicts between git deps & mvn deps (in this case ClojureScript) it should be possible to use :exclusions right? Has anyone else got this working?#2019-07-0119:34dnolenI keep seeing: Error building classpath. Unable to compare versions for org.clojure/clojurescript: {:mvn/version "1.10.516", :deps/manifest :mvn} and {:git/url "#2019-07-0119:35dnolenOdd because -Stree doesn't show org.clojure/clojurescript after my exclusions#2019-07-0119:55alexmiller@dnolen not sure if that msg connects to something above?#2019-07-0119:55dnolento clarify, I got the above error#2019-07-0119:56dnolenthen I ran clj -Stree and I saw ClojureScript coming from maven#2019-07-0119:56dnolenI used -Stree to systematically remove ClojureScript maven artifacts via :exclusions#2019-07-0119:57dnolenstill get the same error#2019-07-0120:02dnolenalso I don't see ClojureScript except for the git one when I run clj -Spath#2019-07-0120:13alexmiller"got the above error" refers to what?#2019-07-0120:19dnolenError building classpath. Unable to compare versions for org.clojure/clojurescript: {:mvn/version "1.10.516", :deps/manifest :mvn} and {:git/url "#2019-07-0120:19dnolen@alexmiller ^#2019-07-0120:22dnolen(was that missing before?)#2019-07-0120:51alexmilleryes :)#2019-07-0120:51alexmillernow I see it, maybe slack being weird#2019-07-0120:52alexmillerthe resolution algorithm may download things that don't show up in the final tree as it goes through the process of walking the deps and making decisions#2019-07-0120:53dnolenI guess I'm confused as it to it barfs#2019-07-0120:53dnolenand doesn't start#2019-07-0120:53dnolenoh but maybe something adds it?#2019-07-0120:53dnolenthough I guess there's no way for me to see what?#2019-07-0120:53alexmillerwell at some point it's encountering the maven version and then trying to decide whether it's newer/older than the git version#2019-07-0120:53alexmilleryou can add a hidden verbose flag#2019-07-0120:54alexmillerclj -Sdeps '{:aliases {:v {:verbose true}}}' -A:v#2019-07-0120:55alexmillernot really designed to be understandable to anyone but me. I'm not sure you'll get the parent child linkage to know where it's coming in from that#2019-07-0120:56alexmillerif you specify the clojurescript as a top dep, it should never need to make a decision though, so I'd expect that to work regardless#2019-07-0120:57alexmillerif you want me to look at it in more depth, send me the full deps.edn and the verbose output if you have it, don't need to spam the channel here with it#2019-07-0120:58dnolenlet me see if I can just make this minimal#2019-07-0120:58dnolenwill drop something later, have to head for now#2019-07-0121:03alexmillernp#2019-07-0208:53ikitommiis there an example out there how to do mixed java + clj projects with deps?#2019-07-0208:57lispyclouds@ikitommi AFAIK tool.deps doesn't support java compilation. Only way I know to get around it is build the java project separately and import it as a dep in deps.edn.#2019-07-0208:59ikitommithanks, but that was not the answer I was hoping for.#2019-07-0209:00OlicalI messed around with JNI stuff and deps.edn but basically did the compiling myself through a Makefile, probably not helpful but still. https://github.com/Olical/snowball#2019-07-0209:00lispycloudsyeah I had asked about it a while back and got the answer. compiling java is beyond tools.deps scope i think and it makes sense#2019-07-0212:32alexmillerMy two suggestions are either to use an additional build tools around clj or to wall off your Java bits in their own project with their own artifact build #2019-07-0212:33sogaiuhaven't tried the following yet, but it looked promising: https://github.com/EwenG/badigeon#2019-07-0213:15seancorfield@ikitommi you could always cheat and use the -e option to run clojure.java.sh and shell out to the Java compiler 😇😂:zany_face:#2019-07-0213:18ikitommigreat idea Sean 😉 did a external compile for my case, but puzzled that has the idea been that people would start using deps instead of boot or lein? Still need a “do-it-yourself” build tool on top - not the most welcoming newbie experience. Some official lein’ish battery-pack would be great?#2019-07-0213:19ghadinewbies don't want to compile java 🙂#2019-07-0213:19ghadibut battery-pack is something we've talked about#2019-07-0215:31ewenbadigeon can indeed compile java, let me know if you have any issue with it#2019-07-0217:28hiredmansomeone in #core-async was having issues using badigeon to build an uberjar and the errors they were getting made me thing badigeon was double aot compiling core.async somehow, but looking at it again, I suspect badigeon may have an issue with timestamps so the clojure source files can end up being tagged with a newer time then the aot'ed code, which can break things#2019-07-0217:32hiredmanmaybe here https://github.com/EwenG/badigeon/blob/master/src/badigeon/jar.clj#L120-L125#2019-07-0511:20ewen@U0NCTKEV8 yes thanks you were right. I forgot to set the last modification time when moving files to a jar/zip archive#2019-07-0215:34TravisI am about to try out badigeon myself for compiling some protobuf java classes#2019-07-0215:49seancorfield@ikitommi Given how many beginners I've seen get tripped up with ~/.lein/profiles.clj in the last few months, I think there's a good argument for simpler tooling like the Clojure CLI. I've seen beginners struggle a lot less with the CLI than you might think.#2019-07-0215:50seancorfieldAnd, yes, as @ghadi says, "newbies" aren't trying to compile Java and Clojure together.#2019-07-0215:51seancorfieldAt work, we do everything with the CLI and a small handful of (sharp) tools. We have a shell script purely to automate repeated sequences of clojure invocations.#2019-07-0216:10dominicm@ikitommi in pack.alpha I do some clever stuff to compile java using the JDK api. Not much better than shelling out 🙂#2019-07-0216:11dominicmI have toyed around with the idea of using the ztellman library for reloadable java with tools.deps but never actually done it.#2019-07-0217:31dnolen@alexmiller circling back - the problem goes away if I supply a ClojureScript git dep version in the root project. Is the behavior I observed yesterday a bug, or just the current behavior? if I supply all the exclusions should I be able to avoid the warning about comparing versions?#2019-07-0217:45alexmillernot sure#2019-07-0217:45alexmillerwalking the full deps tree necessarily requires comparing versions of things, and comparing mvn to git is not (yet) supported (although I know how to do it)#2019-07-0217:46alexmillerthe question is whether in this particular scenario, with exclusions, the comparison can be avoided. exclusions are tricky and it's hard for me to answer that question without having the full scenario in hand.#2019-07-0217:46dnolenk#2019-07-0218:10dnolen@alexmiller ok when I made a minimal case :exclusions worked for me - so I guess -Stree can't be used reliably?#2019-07-0218:28dnolenhrm I guess perhaps a comparison happens before you know it will be excluded?#2019-07-0218:29alexmilleryes, and as you encounter more things, it's possible for the excluded state of something to change#2019-07-0218:29dnolenverbose output just seems to confirm that my exclusions worked#2019-07-0218:30dnolenbut the end behavior is surprising#2019-07-0218:30alexmiller-Stree tells you the end state, but is not the whole story of how it got there#2019-07-0218:30dnolensince for the end user this doesn't seem deterministic#2019-07-0218:30dnolensometimes exclusions work#2019-07-0218:30dnolensometimes it doesn't#2019-07-0218:31dnolenit also breaks using git deps transitively#2019-07-0218:31alexmilleryes, there is a ticket in this area (specifically around cljs iirc, although not with a git dep)#2019-07-0218:31dnolenit can't be done safely#2019-07-0218:31dnolenso you have to re-declare your deps at the bottom#2019-07-0218:31alexmillerI think you are jumping to conclusions that are not necessarily true#2019-07-0218:32dnolenso you have some way to avoid the above?#2019-07-0218:32dnolenI have to re-declare my git deps right now#2019-07-0218:32alexmillerit is possible to declare deps trees that are either logically inconsistent (impossible to satisfy the deps and exclusions as stated across the tree) or that are ambiguous#2019-07-0218:33dnolenbut that's not what I've described above#2019-07-0218:33alexmillerwell you haven't given me a repro, so I can say either way#2019-07-0218:34dnolenyeah I don't know how make this one since it's non-trivial project with non-trivial deps#2019-07-0218:34dnolenit works in the trivial case#2019-07-0218:34alexmillermost things do :)#2019-07-0218:35dnolen... oh#2019-07-0218:35dnolenother git stuff won't appear in -Stree?#2019-07-0218:35alexmiller-Stree should be the full tree of everything chosen at the end (it's just the inverted lib map which you can find in the .cpcache in the .lib file)#2019-07-0218:36alexmillerso other git stuff will appear there#2019-07-0218:36alexmillerit does not list things that were children but not chosen (due to exclusions or version selection or being dominated by a top dep)#2019-07-0218:37dnolenah#2019-07-0218:37alexmillersorry, I have to run to the dentist#2019-07-0218:37dnolenone second maybe I see a way to repro now#2019-07-0218:37dnolenk, will drop a link to something minimal if I can make it#2019-07-0220:19dnolenI found the issue which arose from a misunderstanding on my part#2019-07-0220:20dnolenI had a project which delcared a ClojureScript dep - and of course there were transitive deps that also declared ClojureScript but no conflicts because I guess clj considers the deps.edn :deps to be overrides?#2019-07-0220:21dnolenonce I was using that project transitively - those became conflicts#2019-07-0220:22dnolenI had removed the conflicts from the downstream project, but not this upstream one#2019-07-0220:39alexmillerOn your question above about overrides, I’m not sure what you mean but I suspect you’re making up a model there. Top level deps are always taken as overriding decisions of the transitive deps#2019-07-0220:40dnolenthat's what I meant#2019-07-0220:40alexmillerWhen you move to your “upstream” project though, the deps of the “downstream” project are no longer top deps#2019-07-0220:40dnolen"top-level"#2019-07-0220:40alexmillerSo are subject to other selections#2019-07-0220:40dnolenyeah that was the model I came up w/ based on what I saw#2019-07-0220:40dnolenso good to hear that confirmed#2019-07-0305:51ikitommi@dominicm thanks, alias / sample to pre-compile java classes would be good for many cases. Also enjoying Zach’s hot reload lib (https://github.com/ztellman/virgil) with lein. The latest version has never worked for me thou. Many of our newer libs include Java, because of perf, client projects because of… legacy.#2019-07-0305:52ikitommiAgree that it’s not a most common thing, but still important for the interop story.#2019-07-0305:59ikitommiWhat about the classpath with shell scripting the java compilation? The Java classes might need the project dependencies, including Clojure itself. Resolved that manually just for this case, but would appreciate a sample how to combine full library classpath from deps to feed into a custom javac step. Will check pack.alpha too.#2019-07-0312:15alexmillerclj -Spath ?#2019-07-0308:34dominicm@ikitommi just had an internal discussion about java compilation, the topic of windows came up as problematic for shelling out. If I was investigating this today, I would probably lean on the JDK API again because of that.#2019-07-0503:02seancorfieldIn case you're not in #announcements I just released 0.7.6 of https://github.com/seancorfield/clj-new with suggestions from @dominicm and @tkjone -- thank you!#2019-07-0503:18seancorfieldThis supports multiple templates in a single git repo and makes it easier to have conditional files in a template (`->files` accepts and ignores nil entries).#2019-07-0508:05dominicmOn a similar vein, @viesti just added support to https://github.com/juxt/pack.alpha for building docker images straight from Clojure! With a big thanks to him for that!#2019-07-0508:05dominicm(Now, I need to go and incorporate the clj-new changes into Edge!)#2019-07-0514:20alexmillerclj 1.10.1.462 was released yesterday - no changes in tools.deps, but this undid some of the installer changes so that the install deps.edn is being installed again. this allows older tools.deps lib users relying on the installed clj's reporting of config files via clj -Sdescribe to continue working without update. We saw some issues with this in Datomic ion deploy and Cursive.#2019-07-0521:05cfleming@alexmiller Sorry, I didn’t get time to look at that yesterday.#2019-07-0521:06cflemingI’ll try to get to it soon - it’s the weekend here and I’ve been busy dealing with IntelliJ bugs unfortunately.#2019-07-0522:13cflemingDeps doesn’t support global exclusions, right? i.e. I don’t want a particular dep anywhere.#2019-07-0523:05alexmillerNot yet, no#2019-07-0522:29seancorfieldI've just identified a potential security flaw in seancorfield/depstar 0.2.1/0.2.2/0.2.3 and I have released 0.2.4 to address it. If you are using those earlier versions, please update to the new release a.s.a.p.#2019-07-0603:43dominicm@seancorfield what's the security issue?#2019-07-0604:09seancorfield@dominicm The PR I accepted for 0.2.1 allows for depstar to tree walk through the file system and overwrite files outside the temporary directory.#2019-07-0604:11seancorfieldI didn't realize until it failed to build an uberjar on our CI system -- which had always worked before, strangely -- but failed once it tried to create the parent directory (new in 0.2.1).#2019-07-0604:13seancorfield(so I'm not entirely certain the bug was introduced in 0.2.1 but that was the build that first caused "random" directories to be created if you tree-walked, which wasn't good)#2019-07-0604:13alexmillerand yet you want Clojure to create classes directories... ;)#2019-07-0604:14seancorfieldThe security issue isn't technically the auto-creation of the directory...#2019-07-0604:15seancorfield...I suspect the bug's been there for quite a few releases prior but the behavior was masked until the mkdirs on the parent folder was added.#2019-07-0604:18seancorfield(and the behavior was somewhat benign before 0.2.1)#2019-07-0613:12geraldodev#2019-07-0613:12geraldodevI'm using reply (the one that comes with lein) and it initiates a nrepl server . Problem was the it starts nrepl in a way that it does not write .nrepl-port file that my beloved Fireplace (vim) needs to connect to nrepl. In case you have the same problem I'm asking reply at init to do this by using the snippet attached#2019-07-0613:24geraldodev@seancorfield I'm using your dotclojure file. the reply section was made by you. I've checked today and you do not have reply anymore. What repl client are you using ?#2019-07-0616:36seancorfieldI don't remember ever having reply in there @geraldodev but it does have both the old and new nREPL aliases. I use a Socket REPL and Chlorine for Atom for editing or unravel/unrepl from the command line. #2019-07-0617:47geraldodev@seancorfield you are right I've probably associated with you because I've forked https://github.com/seancorfield/dot-clojure#2019-07-0618:40dominicm@geraldodev do you use the reply in the terminal? If not I would recommend using nrepl.cmdline, which does write the .nrepl-port file (and deletes it afterwards).#2019-07-0620:45geraldodev@dominicm I use reply on terminal. Since I've looked up reply on sean's dot-clojure I've seen this nrepl.cmline there and I've invoked to see if it would be less complicated than the eval parameter that I'm passing to reply. But no, it writes .nrepl-port but I would have to invoke reply (the client) as another proccess which I was doing before. Invoking reply directly it 1) starts nrepl 2) writes .nrepl-port and Fireplace can consume this. I get one less java process. I dealt with this problem because I'm working with 2 repls . One for server and another (swing) for client. That would be 4 process and one more tmux screen.#2019-07-0621:16dominicmI'm a fellow fireplace user, I understand, mostly! I don't find myself reaching for the terminal repl at all.#2019-07-0713:08dharriganWas there discussion around deps.edn supporting alias(es) of aliases, or does that go too far?#2019-07-0805:56lepistanei think it already does#2019-07-0806:48dharriganI'm not seeing that described in the deps_and_cli documentation#2019-07-0806:48dharrigan(in the aliases section)#2019-07-0806:51yogidevbearI don't think it discusses them directly, but there are alias references in the examples https://clojure.org/guides/deps_and_cli#2019-07-0806:51yogidevbearhttps://clojure.org/reference/deps_and_cli#_aliases#2019-07-0806:52yogidevbearThen again ☝️#2019-07-0806:52dharriganYes, I read that#2019-07-0806:52dharriganbut nowhere does it say, or imply or give example that an alias can reference another alias#2019-07-0806:52dharrigani.e.,#2019-07-0806:52yogidevbearOh I'm with you now#2019-07-0806:53dharrigan{:aliases {:foo ....
:bar :foo}#2019-07-0806:53dharriganor something like that#2019-07-0806:53dharriganwhere bar can pull in the configuration of foo#2019-07-0806:53dharriganso intead of having a big command line like#2019-07-0806:54dharriganclj -A:repl:bar:test:foo you can do cli -A:super-aliases-to-the-max that is the aggregation of repl:bar:test:foo#2019-07-0806:55yogidevbearLol, love the choice of naming -A:super-aliases-to-the-max#2019-07-0806:55dharrigan🙂#2019-07-0807:04yogidevbearI might be reading the source incorrectly, but looking through the t.d.a reader.clj, I don't think alias of aliases is supported. Disclaimer: insert /giphy I don't know what I'm doing 😉#2019-07-0808:14dominicmInheritance is not supported. I don't think it's going to be, but I'm not qualified to make the assertion on the latest thinking.#2019-07-0808:25sogaiuafter a while, generating one's deps.edn from something else starts to seem attractive 🙂#2019-07-0809:06kirill.salykinhi,
I am trying to add athena-jdbc jar as local/root
but without any success
[ERROR] 'artifactId' with value 'AthenaJDBC${env.JDBC_V}' does not match a valid id pattern. @
#2019-07-0809:06kirill.salykinIs this something wrong with pom file?#2019-07-0809:07kirill.salykin<groupId>Athena</groupId>
<artifactId>AthenaJDBC${env.JDBC_V}</artifactId>
<version>${env.MAJOR_V}.${env.MINOR_V}.${env.REVISION_V}.${env.BUILD_V}</version>
<packaging>jar</packaging>
<name>AthenaJDBC${env.JDBC_V}</name>
<url>http://maven.apache.org</url>
#2019-07-0809:07kirill.salykinplease advice#2019-07-0809:33dharrigan@dominicm I wonder why not supported? I'm sure it was thought about/discussed/hammocked.#2019-07-0820:01felipebarrosFor that use case, why not just use the alias functionality of your shell? It may not be ideal but it will surely do what you want. max-alias RET.#2019-07-1210:56dharriganTrue true#2019-07-0817:42seancorfield@kirill.salykin That pom.xml file is expecting you to have environment variables set to specify all those variables. If you have that as a local dep, I suspect that the project is expecting the final pom to be built as part of the build/deploy process -- and does not expect you to try to use the project from source.#2019-07-0817:44seancorfieldYou could probably use this version direct from Maven https://search.maven.org/artifact/io.burt/athena-jdbc/0.1.0/jar -- that would be a regular remote dep.#2019-07-0818:35kirill.salykinThanks! You are the mvp, was a bit surprised that jar contains half baked pom#2019-07-0915:04y.khmelevskiiHi gents! Can you please help me to understand how I can reload duct server when I change required library
I use deps.edn together with lein
{:paths ["src"]
:deps
{...
shared {:local/root "../shared"}}
and when I change files in shared dependency I can’t apply them#2019-07-0915:05kirill.salykinI might be mistaken, but I think you should restart the repl#2019-07-0915:05kirill.salykinso deps.edn can rebuild classpath#2019-07-0915:05kirill.salykinthere was something which supports dynamic load
add-lib I think, but dunno much about it#2019-07-0916:08y.khmelevskiiHmm, it’s not good idea to restart repl every time when I changed shared dependency#2019-07-0916:15seancorfieldThere's nothing different about using deps.edn vs lein in terms of reloading your app while you're working. I do it all the time with our Component-based apps: stop the system, reload any namespaces you need to, start the system again.#2019-07-0916:17seancorfield(although, to be honest, I almost never need to do that as I eval code into the running image all the time while I'm working, and I use Var references for a few key things around routes and middleware so I don't need to restart the app)#2019-07-0916:20kszabo@y.khmelevskii in case of local dependencies, there’s no need to restart the jvm, since the dep folders are on your classpath you can just use cider-ns-reload-all or your editor’s equivalent to get the latest state#2019-07-0916:22kirill.salykinOh, I misunderstand you
If it is already on the class path - you just need reload tell, not restart#2019-07-0916:26kennyA bit of a tangent... it'd be useful to be able to switch from a git/maven dependency to a local one at runtime. That would remove my primary reason for needing to restart the REPL.#2019-07-0916:59alexmillerThat’s not a thing that would be possible to do in general for stuff that’s already loaded#2019-07-0919:39dominicmIt's not so bad when combined with t.n.s#2019-07-0919:39dominicmYou can run a refresh afterwards.#2019-07-0916:27ghadinot possible#2019-07-0916:29kennyFor all dep type cases or just maven?#2019-07-0916:32seancorfieldI strongly recommend developing a tight REPL-Driven Development workflow where you eval code as you change it -- small, fast feedback loop -- rather than making a bunch of changes and then trying to restart/reload your app. Then these sorts of questions just don't even come up. I start my REPL/REBL combo up and tend to leave it running for days, sometimes even a week or more, and I might only need to restart my app once or twice a week.#2019-07-0916:32kirill.salykin@y.khmelevskii maybe this can be useful https://lambdaisland.com/blog/2018-02-09-reloading-woes#2019-07-0916:33kenny@seancorfield Not sure how that would solve this problem. The dependencies are immutable so you cannot make changes to them.#2019-07-0916:34seancorfieldHow often do you find yourself changing dependencies? (you can add new dependencies on the fly via the add-lib branch of tools.deps)#2019-07-0916:36seancorfieldWe have an 80K+ line code base at work and dozens and dozens of external lib dependencies -- and we update them in a controlled manner every few weeks or so.#2019-07-0916:36kennyOften. Currently we have ~15 internal libraries that a particular service may depend on. They are all git deps. If you find a bug or need to add something to a particular library, you need to stop your repl and then decide to either change the library dep to a local/root or push a new version of the lib.#2019-07-0916:37kennyIt's not usually adding a dependency - that's rare. It's almost always updating to a newer version or switching to local/root.#2019-07-0916:41seancorfieldInteresting. We don't run into that because we have a monorepo -- so all code is already a local dep. We can edit any of it or pull new versions down and, at worst, just reload all on the main namespace of the subproject we're working on.#2019-07-0916:42seancorfield(one of the main drivers for our switch from Boot to the CLI tools was that it would work better with our monorepo -- because we were having problems with Boot's pods and the shadow fileset thing it does)#2019-07-0916:45kennyYeah I have been seriously considering switching us to a monorepo to remove this pain point. I hate having to restart the REPL to go and fix a small bug.#2019-07-0916:46kennyI've also thought about some sort of hackery that would temporarily update the .gitlibs file to a local/root file so you could dynamically switch between a git dep and a local one. Haven't written anything around that concept yet though. It feels a bit icky but so is restarting the REPL a lot.#2019-07-0916:48mccraigmccraig@kenny we used to have multiple repos and switched to a monorepo - so much better#2019-07-0916:49kenny@mccraigmccraig @seancorfield How do you guys deal with ensuring tests/build steps for a particular project only run when that project changes?#2019-07-0916:50mccraigmccraig@kenny we don't . we run CI across all modules in the monorepo on every commit#2019-07-0916:50kennyAh. That seems tricky. That could easily result in extremely long build times for simple changes.#2019-07-0916:53mccraigmccraigsure, a CI run is 10 mins for us, but it doesn't matter - your local dev loop is generally to run tests in one namespace from the editor and is lightning fast#2019-07-0916:54kenny10 mins would be ok but I don't think we'd be close to that. Many services need to build docker images, run integration tests, build cljs, and run hundreds of thousands of gen tests. I could see the build time easily going over an hour.#2019-07-0916:57mccraigmccraigyou could look at the paths changed by the last commit and only run tests in changed modules#2019-07-0916:58mccraigmccraigalthough presumably you would also want a full run sometimes#2019-07-0917:01kennySomething like that may work. Though there doesn't appear to be any tooling around this sort of stuff yet.#2019-07-0917:04kennyI've been very close to digging in on that front because of how painful the REPL restarts are. A monorepo seems like quite a useful tool for development but it also feels like it may open a different can of worms. Blog posts around this topic would be very useful 🙂#2019-07-0917:25seancorfield@kenny (sorry, stepped away) I mostly run tests directly from the editor while I'm developing but we can run tests selectively for any combination of subprojects locally from the command line or a full suite end-to-end for a CI-style sweep. Again, tho', working in a tight feedback loop -- eval'ing code as you write it and either running test expressions from Rich Comment Forms or actual tests via the editor -- means that your risk of breaking anything is minimal.#2019-07-0917:40kenny@seancorfield Right - the dev story is great. I was talking about CI time for a monorepo and how a single push could easily result in very long build times.#2019-07-0918:26ghadi@kenny while developing locally on 15 dependent libs, I'd definitely use local/root#2019-07-0918:27ghadiExclusively#2019-07-0918:28ghadiBecause of the bug fix scenario you described#2019-07-0918:28kennyYou'd then need to guarantee that all the devs on your team have the same directory structure, I'd think.#2019-07-0918:28ghadiNo, just generate the deps file#2019-07-0918:29ghadiIt's data for a reason#2019-07-0918:29kennyNot sure what you mean. Also, I think that'd lose the valuable comments that we have in our deps.edn.#2019-07-0918:31ghadiI mean: programmatically output the local deps file with some source file#2019-07-0918:31ghadiCan comment the source file#2019-07-0918:32kennyOh, interesting#2019-07-0918:32seancorfieldWe auto-generate an everything/deps.edn file from all the other deps.edn files (each subproject has its own) and then we generally run a REPL/REBL based on that "everything" deps file.#2019-07-0918:32ghadi^^#2019-07-0918:33ghadiIf you use git deps locally, that conflicts with manual iteration on the dependency#2019-07-0918:33ghadiA git SHA is a value, local dir is a mutable zone#2019-07-0918:33kennySo you have a deps-base.edn (or something like that) and then when you spin up a clj process, you generate a deps.edn based on the "environment" you want?#2019-07-0918:33ghadiYeah#2019-07-0918:33ghadiOr generate it one time for a developer onboarding#2019-07-0918:34ghadiOr whenever the source file changes#2019-07-0918:34ghadiExactly#2019-07-0918:35kennyThat seems doable and would remove the need for a monorepo. This seems like very valuable info that should be a part of the deps.edn doc if it's not already.#2019-07-0918:37ghadimonorepo is a big hammer#2019-07-0918:38ghadijust a little read => update => spit is enough to avoid that hammer#2019-07-0918:41seancorfield> This seems like very valuable info that should be a part of the deps.edn doc if it's not already.
I think different people have different approaches -- I don't think you can generalize this sort of thing for the base set of documentation.#2019-07-0918:42kennyTrue. An outline of various approaches you could use to solve these problems in the real world would be fitting.#2019-07-1006:24just.sultanovHi folks! In branch add-lib there is a corrected spec :local/coord, but in the branch master is absent. So it should be? https://github.com/clojure/tools.deps.alpha/commit/458293aaf0e719e896bddb895e2f7d47aa69637a#2019-07-1108:18borkdudelately I'm getting
Full report at:
/var/folders/2m/h3cvrr1x4296p315vbk7m32c0000gp/T/clojure-6407140529537279679.edn
when I get an exception running a program with tools.deps. Is there a way to get the old behavior back?#2019-07-1108:21kirill.salykinhttps://clojure.org/reference/repl_and_main#_as_launcher#2019-07-1108:22kirill.salykin--report=stderr
I think#2019-07-1108:22kirill.salykin@borkdude#2019-07-1108:24borkdude> file - write to a temp file (default, falls back to stderr)
when would this "fall back to stderr" as a default?#2019-07-1108:24borkdudeif it has no write access?#2019-07-1108:29borkdudeThis doesn't work for me:
clj --report stderr -A:clj-kondo --lint corpus/unresolved_symbol.clj
#2019-07-1108:29borkdudeIt says "-A:clj-kondo (No such file or directory)"#2019-07-1108:33kirill.salykinmaybe try with -A:clj-kondo in the end?#2019-07-1108:33borkdude--lint ... are arguments to clj-kondo#2019-07-1108:38borkdudeit worked with the java property#2019-07-1108:45vlaaad--report stderr are clojure.main arguments right?#2019-07-1108:46vlaaadmaybe you need to modify :clj-kondo alias's :main-opts and put --report stderr there?#2019-07-1108:46borkdudeyeah, that makes sense#2019-07-1210:17dominicmI think that a JVM option variant was added too, for leiningen.#2019-07-1210:18borkdudeI'm using that right now, works#2019-07-1210:20dominicm> Specifically, the default behavior for error reporting with Clojure 1.10.1-beta3 is to write the error report and stack trace to a file, but you can also change that to stderr via system property -Dclojure.main.report=stderr if you prefer.#2019-07-1210:21borkdudeyes, that one#2019-07-1108:46kirill.salykinseems reasonable#2019-07-1114:31kirill.salykinIf I copy jar to .m2 and add it to deps.edn - it works
mkdir -p ~/.m2/repository/athena/athena-jdbc/2.0.7/
wget -O ~/.m2/repository/athena/athena-jdbc/2.0.7/athena-jdbc-2.0.7.jar
:deps {athena/athena-jdbc {:mvn/version "2.0.7"}
but if I add it via local/root - tools.deps starts to complain about wrong data in pom
:deps {athena/athena-jdbc {:local/root “lib/athena-jdbc.jar”}
why is such difference? is there a way to add jar as from .m2 but from within the project?#2019-07-1114:49ghadiif you get an error, can you post it in a snippet (COMMAND-SHIFT-ENTER) @kirill.salykin#2019-07-1114:49kirill.salykinyes, one moment#2019-07-1114:49ghadiI would like to see the error from when you add it via :local/root#2019-07-1114:53kirill.salykin#2019-07-1114:53kirill.salykin@ghadi ^#2019-07-1114:53ghadiok this is what I suspected#2019-07-1114:54ghadiwhen tools.deps uses a local jar with :local/root && the jar contains an internal pom.xml, it will try to read the pom and fetch dependencies#2019-07-1114:54kirill.salykinshall I remove all poms?#2019-07-1114:54ghadiyeah you could#2019-07-1114:55kirill.salykinand why it doesnt try to do it when reading from .m2?#2019-07-1114:55ghadionly one pom will matter, I forget the exact path#2019-07-1114:55ghadiit will try to read the pom in .m2 - but not the pom within the jar, the one next to the jar if present#2019-07-1114:55ghadiMaybe ^, I forget the exact details#2019-07-1114:55kirill.salykinthere is no pom with the jar#2019-07-1114:56ghadiBut anyways, if it goes through the .m2 path that's a different handler entirely -- uses Maven Resolver#2019-07-1114:56kirill.salykinonly this published by aws
{:deps {jdbc {:local/root "...", :ignore-pom? true}}}#2019-07-1114:58ghadiI'll have to check with Alex about why it does that in the first place#2019-07-1114:58ghadibefore making solutions#2019-07-1114:58kirill.salykinof course, just a little proposal#2019-07-1114:59kirill.salykinthanks a lot for help#2019-07-1114:59ghadinp#2019-07-1114:59ghadihttps://twitter.com/stuarthalloway/status/1148314535830413312#2019-07-1114:59ghadi😂 been pasting that everywhere ^#2019-07-1115:03kirill.salykin)#2019-07-1115:42seancorfieldCan you specify the JAR in :paths and not worry about dependency resolution?#2019-07-1115:48ghadi^^^ oh of course#2019-07-1115:48ghadi@kirill.salykin#2019-07-1116:05kirill.salykinOh, I ll try, thanks!#2019-07-1116:12alexmillerIf you’re using a jar, that has transitive dependencies, how else are you going to satisfy those deps?#2019-07-1116:13ghadimost of them don't have transitive deps @alexmiller -- the usecase is usually downloading some database driver that is self-contained in the jar#2019-07-1116:14ghadiif you were using bare java, you'd put it on the classpath directly#2019-07-1116:15ghadiwhich means, use :paths#2019-07-1116:16alexmilleryes, tools.deps purpose is to build out transitive deps, and it will do so with local jars as well. The no deps case is kind of the base case and it doesn’t matter#2019-07-1116:41kirill.salykinThanks for clarification #2019-07-1216:05kenny@ghadi We talked on Tuesday (https://clojurians.slack.com/archives/C6QH853H8/p1562697498052500) about a tool that would "localize" (i.e. git/url -> local/root) certain dependencies before starting a REPL (or some other task). This would solve the problem of having a single project that depends on lots of internal libraries that often need to be developed in parallel with the project you are working on. You said "monorepo is a big hammer" and to take a "`read` => update => spit" approach. The general approach makes sense but there are a few issues I immediately ran into while trying to work on this tool. I'm curious how you solved these for your company (it sounded like you guys have a tool that does this too).
When you "localize" dep libraries, how do you ensure that the library at a give local/root path is up to date with VCS? Often my local libraries are not up to date with the version on master. You could run a git check before localizing and warn the user that the code at a given local/root is not up to date. Similarly, should you ensure your localized dep libraries are all on the same branch? I could easily see a situation where the local/root of a lib is on a different branch (maybe you started some work there and didn't switch back to master) than the other libs or the service itself that you are working on.
Both of those issues could easily cause strange behavior to occur which could be costly when debugging because you didn't have know the lib was not up to date or on the wrong branch. Having a monorepo would solve both of these problems (it introduces another set of problems though).#2019-07-1217:28dnolenclojure -J-Xms4g -J-Xmx8g ...#2019-07-1217:28dnolenthis should set the JVM memory settings right?#2019-07-1217:29dnolenI keep seeing OOMs from JGIT - is there a different way to set the memory for the tools deps itself?#2019-07-1217:33dnolendoes clojure have it's own memory settings that have to be set independently?#2019-07-1217:34andy.fingerhutOne way is to create an alias in a deps.edn file e.g. deep merge this into deps.edn map: {:aliases {:bigmem {:jvm-opts ["-Xms4g" "-Xmx8g"]}}}. Then run clojure -A:bigmem#2019-07-1217:35dnolen@andy.fingerhut so are you saying what I'm trying above won't work?#2019-07-1217:37dnolennote the OOMs are from clojure tool trying to fetch git deps#2019-07-1217:37dnolenno from launching any stuff I'm trying to run#2019-07-1217:37andy.fingerhutOh, no, I believe it will work. I was responding more to the question I thought you were asking about a way to have settings for this.#2019-07-1217:37dnolenI guess I'm skeptical that 8g wouldn't be enough for JGIT#2019-07-1217:37andy.fingerhutAnd I probably misunderstood which JVM you wanted those options to go to, now that I read your question again.#2019-07-1217:38dnolenyes not my own process#2019-07-1217:38dnolenI want clojure to not fail fetching git deps#2019-07-1217:40dnolenoh erg#2019-07-1217:41dnolen@alexmiller it looks like clojure has a hardcoded memory setting of -Xmx256m??#2019-07-1217:41dnolenI'm using clojure in a CI context, and it consistently fails#2019-07-1217:42andy.fingerhutYeah, that is what I see in my /usr/local/bin/clojure shell script, too.#2019-07-1217:55alexmillerIt’s hard coded for the process that builds the classpath#2019-07-1217:56alexmillerThis is the first report I’ve seen of an issue with that but happy to mod#2019-07-1217:57alexmillerCurious what it’s doing when it fails. Maybe a git version compare?#2019-07-1218:00dnolen@alexmiller PMed the output#2019-07-1218:01dnolenI thought I could set this myself with JAVA_CMD#2019-07-1218:01dnolenbut no good because of the hard-coding at the moment#2019-07-1218:02dnolenI hacked it by seding clojure shell script on the CI instance#2019-07-1318:16yuhanIs it possible to mix Java source files in a deps.edn project?#2019-07-1318:17andy.fingerhutI believe the answer is that you can have Java source files, but deps.edn isn't intended to do anything with them.#2019-07-1318:18yuhanso I just put them in the src/ folder and they'll get picked up by an (import) statement?#2019-07-1318:20andy.fingerhutSomething (other than the clj or clojure commands) is welcome to compile Java source files to .class and/or .jar files that can be on a classpath directory specified in a deps.edn file, but having such source files around does not cause clj or clojure commands to compile them.#2019-07-1318:20andy.fingerhutYou should use some other tool to cause Java source files to be compiled.#2019-07-1318:23yuhanoh I see, that clears things up - so I'll need to compile a .jar file and refer to it using a :local/root dep#2019-07-1318:23andy.fingerhutThat is one way, yes.#2019-07-1318:26yuhanhmm, I guess that's what Lein is doing with its :java-source-paths key#2019-07-1318:27yuhanJust to confirm, does that work with interactive development - if I recompile the jar and refresh the clojure ns in the REPL, will the updated classes be imported?#2019-07-1318:35andy.fingerhutI have not tried, but I would guess no, with the default choice of JVM class loaders used by Clojure. Maybe there is a way using non-default classloaders to make something like that happen, but I have never done it, and I could be wrong in guessing that it is even possible. You would likely be asking for additional headaches by attempting to do so.#2019-07-1318:38andy.fingerhutIf you like the idea of diving in and seeing if you can make it work, this StackOverflow was an early hit I found for Google search terms: "java reload class from jar" https://stackoverflow.com/questions/728140/can-i-dynamically-unload-and-reload-other-versions-of-the-same-jar#2019-07-1318:39andy.fingerhutBut I would strongly caution you that without more expert help than me (or perhaps you have already done similar things before), you could be in for a lot of confusing debugging and frustration there.#2019-07-1318:46yuhanThanks for the advice - I'll probably hold off on doing anything like that then, having little experience with JVM side of things#2019-07-1616:31credulousHi. I’m a bit confused with my workflow using clj and deps.edn. Everything was so magic with lein, I never had to actually understand what was going on. 🙂 no such luxury/laziness here.#2019-07-1616:31credulousI have an app that has both an API server and a clojurescript frontend. The server is using http-kit.#2019-07-1616:32credulousIn my deps.edn I have {:aliases {:run {:main-opts [ "-m" "alexandria.core" ] } } — in alexandria.core I initialize the db, start the http-kit server etc.#2019-07-1616:32credulousIn my global deps.edn, I have the alias :nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.6.0"}}}#2019-07-1616:33credulousWhat I would like to do is start the server, and have an nrepl session start at the same time, preferably with both a repl prompt and a server to connect to from the editor (I use fireplace/vim)#2019-07-1616:34credulousRight now I’m starting the server with clj -A:run, which doesn’t start a repl as near as I can tell#2019-07-1616:40alexmillercorrect#2019-07-1616:34credulousand in a different terminal clj -R:nrepl -m nrepl.cmdline, which starts a nrepl server#2019-07-1616:41alexmillerprob correct#2019-07-1616:35credulousand in a different terminal just clj to get a repl prompt#2019-07-1616:42alexmilleryou might need something more here to get an nrepl client, not just a repl ?#2019-07-1616:35credulousand in a different terminal shadow-cljs watch alexandria to get clojurescript compiling plus a clojurescript repl going#2019-07-1616:35credulousI’m sure this is not the Right Way to do things.#2019-07-1616:43alexmilleryou could probably write a tiny script that combines several of these things and call that instead#2019-07-1616:43alexmillerand I'm not sure if you need the clj at all?#2019-07-1707:16dominicm@credulous one problem here is that your nrepl is a different jvm to your main application. So when you eval in nrepl, it won't be reflected in your web server.#2019-07-1714:40alexmillerclj 1.10.1.466 / tools.deps.alpha 0.7.527 released:
- fix TDEPS-134 - respect Maven mirror settings in ~/.m2/settings.xml
- change error on unknown aliases to a warning
- deprecate and warn on use of clojure.tools.deps.alpha.reader/clojure-env
- change default JVM options when using tools.deps to build classpath#2019-07-1718:53dominicmI'm glad it's a warning again, I can now just spam the known common dev aliases instead of caring whether they exist 😄#2019-07-1814:06borkdudehttps://twitter.com/borkdude/status/1151855775956066311 🙂#2019-07-1821:12kenny@seancorfield I updated my CI to openjdk-11 from openjdk-8 and my uberjar step using depstar has started failing with some sort of memory issue. The uberjar is built with clojure -J-Xmx3g -A:uberjar -v and the command exits with Killed and the status code 137, which typically indicates a memory issue. I'm using the latest version 0.2.4. Any thoughts on why depstar would have a memory issue on Java 11?#2019-07-1821:14kennyI am using the -v option for depstar. It's interesting that the build always hangs on this file /home/circleci/.m2/repository/http-kit/http-kit/2.3.0/http-kit-2.3.0.jar for a few seconds right before the Killed message.#2019-07-1821:16ghadiDid you set a max heap limit for depstar#2019-07-1821:17ghadiDid you look up what status 137 means?#2019-07-1821:17ghadiAh you did set Max heap#2019-07-1821:18kennyIt's happening on CI and according to CircleCI that usually means a mem issue: https://circleci.com/docs/2.0/java-oom/#debugging-java-oom-errors#2019-07-1821:19kennyIt's strange that it's happening at all. I can't imagine it using close to 3g of mem.#2019-07-1821:19kennyIt almost makes me think the -Xmx is getting ignored for some reason.#2019-07-1821:20ghadiSet it to 1GB#2019-07-1821:21kennySame issue#2019-07-1821:21ghadiI think there was a change where depstar now unzips in memory#2019-07-1821:21kennyAh, that sounds relevant. I'll try an older version.#2019-07-1821:24kennyIt works with 0.1.7:
clojure -J-Xmx3g -Sdeps '{:deps {seancorfield/depstar {:mvn/version "0.1.7"}}}' -m hf.depstar.uberjar MyProject.jar
#2019-07-1821:25kennyIt's far slower though#2019-07-1821:27kennyThe code of depstar is small, should be fairly easy to find the issue. Must have something to do with Java 8->11.#2019-07-1821:33ghadiIt was smaller before Sean fixed all my bugs#2019-07-1821:36kennyThis doesn't sound relevant but I found this https://bugs.openjdk.java.net/browse/JDK-8222532 which is resolved in Java 11.0.4. I'm running 11.0.3.#2019-07-1821:39kennyIt's not obvious how to update to 11.0.4 for some reason. Maybe the debian packages haven't updated or something.#2019-07-1821:44ghadidunno I use adoptopenjdk#2019-07-1822:05kennyI don't think 11.0.4 is released yet.#2019-07-1822:10kennyIt's possible that issue is relevant but it's proving hard to validate given 11.0.4 isn't out yet. That issue may not even be the problem. Going to roll back to Java 8 for now.#2019-07-1822:17ghadiit's out#2019-07-1822:17ghadiadoptopenjdk has it#2019-07-1822:17ghadi@U083D6HK9#2019-07-1822:19kennyThere doesn't appear to be a debian package yet. Also couldn't find a command to force update to 11.0.4. All guides seem to suggest using an install path that doesn't let you select your version.
I don't see a 11.0.4 version on their docker page yet: https://hub.docker.com/r/adoptopenjdk/openjdk11#2019-07-1822:19kennyI see 11.0.4 here though: https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=hotspot#2019-07-1822:44seancorfieldThe change with depstar was from expanding everything out to a temporary folder and then copying it all back into a ZIP (JAR) to just copying everything into a ZIP (JAR) directly -- which is why 0.1.7 is so much slower than 0.2.x. It now uses a zip filesystem directly. I wasn't aware that changed the memory usage but it's certainly possible. All our uberjars are 30-40MB each and we haven't seen any memory issues using the default heap settings running clj...#2019-07-1822:45kennyThe uberjar for the service with this issue is 60mb.#2019-07-1822:47kennyIt definitely has to do with java 11. I reverted our build and prod images to java 8 and everything builds as expected - no mem issue.#2019-07-1822:47seancorfieldThe change also served as a solution for Windows-related issues with reserved filenames (so you couldn't run depstar 0.1.7 at all on Windows when you depended on libraries that contained certain filenames!).#2019-07-1822:48seancorfieldGood to know. We're on Adopt OpenJDK8 everywhere because some stuff we rely on won't run on JDK11.#2019-07-1822:51kennyWe've been using CircleCI's pre-built images for building uberjars. Specifically circleci/clojure:openjdk-11-tools-deps-1.10.0.442 is the one that produces the mem issue. Not sure if they use adopt openjdk or not.#2019-07-1908:20just.sultanov@U083D6HK9 we use the latest versions and the project works on openjdk 8,9,11,12:
http-kit {:mvn/version "2.4.0-alpha4"} ;; fixes java 11
javax.xml.bind/jaxb-api {: mvn / version "2.3.1"} ;; support java 9+
org.clojure/core.rrb-vector {: mvn / version "0.0.14"} ;; support java 11+
see more info about http-kit: https://github.com/http-kit/http-kit/releases/tag/v2.4.0-alpha3#2019-07-1915:24kenny@U1EQNSHL4 Not sure I understand what you mean. This issue was about building an uberjar with depstar, not an issue with http-kit.#2019-07-2001:05steveb8nQ: has anyone seen and fixed this warning “SLF4J: Class path contains multiple SLF4J bindings.”#2019-07-2001:05steveb8nfor the life of me I can’t figure out how to add the deps exclusion correctly in deps.edn#2019-07-2002:50alexmillerIn the coordinate, add :exclusions [group/artifact]#2019-07-2006:31steveb8nthanks but I need to know which dep to exclude. I should have been clearer in my question.#2019-07-2006:31steveb8nhere’s the warning I see
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/steve/.m2/repository/com/fzakaria/slf4j-timbre/0.3.13/slf4j-timbre-0.3.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/steve/.m2/repository/org/slf4j/slf4j-nop/1.7.7/slf4j-nop-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [com.github.fzakaria.slf4j.timbre.TimbreLoggerFactory]
#2019-07-2006:34steveb8nI’m want to use timbre so this suggests I should exclude the slf4j-nop dep. when I examine my deps tree, that is a transitive dep from Clojure (via tools.deps). I can’t figure out how to exclude that completely, or if there’s a better way to do this?#2019-07-2008:06dominicm@U0510KXTU something is wrong if your clojure depends on slf4j-nop#2019-07-2008:06dominicmdo you mean something other than org.clojure/clojure?#2019-07-2008:07steveb8nlet me double-check….#2019-07-2008:08steveb8nquite right, it’s coming from datomic-free#2019-07-2008:09dominicmdatomic-free probably shouldn't depend on slf4j-nop, but yeah.#2019-07-2008:11steveb8nbut there’s two more: from tools.gitlibs and tools.deps#2019-07-2008:12steveb8nI can’t see how they are getting in there#2019-07-2008:13dominicmtools deps has a transitive slf4j dependency, which I've flagged up that it shouldn't have in https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-108?filter=allopenissues#2019-07-2008:13steveb8nI take it back. the extra one comes as part of tools.deps#2019-07-2008:14steveb8nis there a way to explicitely exclude that? maybe explicitely add tools.deps as a dep?#2019-07-2008:15dominicmI think you just need to change your t.d.a dependency to have an exclusion of slf4j-nop?#2019-07-2008:16steveb8nI didn’t have a t.d.a dep in my deps#2019-07-2008:16dominicmWhere's it coming from?#2019-07-2008:16steveb8nI added one with the exlusion (same as datomic-free) but it’s still there for some reason. digging….#2019-07-2008:17dominicmis this a cursive issue, i.e. does it not appear in clojure -Stree?#2019-07-2008:18steveb8nI think it might be. good idea, I’ll check from the CLI#2019-07-2008:19steveb8nquite right, not a problem from the CLI#2019-07-2008:19dominicmI guess cursive is adding t.d.a to your deps for some reason?#2019-07-2008:20steveb8nyou legend! despite the presence in Cursive, the warning has gone at load time. that fixed it#2019-07-2008:21steveb8nonce again, you’ve helped me. thanks mate#2019-07-2008:21steveb8nif you are at Clojutre, I’ll buy you a beer#2019-07-2008:28dominicmUnfortunately not, I'm at Heart of Clojure though on the chance you are, I know you struggle to get up for the european ones. I'm sure my fellow JUXTers will enjoy the beer though 🙂#2019-07-2008:39steveb8nI’ll pay it forward. I just listened to the Juxt-cast today and heard about XT20. might have to switch to that next year#2019-07-2318:45ticeanIs there a way to control HTTP timeouts or retries with Maven dependencies when using tools-deps? I must have a shakey connection because I’m getting HTTP errors when performing clojure -Stree.
With enough retries I can get them all, but this isn’t working very well my build process. Any suggestions?#2019-07-2318:47alexmillerno support for that, sorry#2019-07-2318:48alexmillerI'm not even sure what Maven offers in the apis for that, but nothing is surfaced through tools.deps#2019-07-2318:53ticeanOk, thanks for the response! Google tells me that there’s some Maven HTTP configurations through XML, but I have very little knowledge in this area. Most of my Maven interactions have been through Clojure tooling. 🙂#2019-07-2318:53ticeanI’ll try in the build server. Hopefully the probs are local to my network.#2019-07-2318:55alexmillerXML in pom.xml or in settings.xml?#2019-07-2318:57ticeanIn settings.xml. If I understand correctly? https://maven.apache.org/guides/mini/guide-http-settings.html#Connection_Timeouts#2019-07-2318:59alexmilleryes, that's settings.xml, but this is for wagons, which is the "old stuff". tools.deps uses the newer Maven resolver with transports instead#2019-07-2318:59alexmillerI doubt anything we're doing in tools.deps is looking at it right now (although honestly I'm not certain)#2019-07-2319:01ticeanAh, I see. Thanks. I figured I was not getting something because I don’t know the ecosystem very well.#2019-07-2319:02alexmillerwe're using org.eclipse.aether.transport.http.HttpTransporter, which does seem to have some sort of configuration facility for http parameters. how to actually get that configured would take some spelunking#2019-07-2319:04alexmillerI kinda see how it could be done#2019-07-2319:05alexmilleranyhow, if you want to file a ticket to request this, you could do so at https://clojure.atlassian.net/servicedesk/customer/portal/1 - this is the first time I've seen someone ask for it, so pretty far down in the queue#2019-07-2319:08ticean👍 Thanks. That’s understandable. Dependency pulls have worked well for me too, until my summer relocation to a cabin in the woods. Has been great for hammocking, now just trying to get those dependencies. 🙂#2019-07-2319:19alexmilleryou just need to mirror maven central and clojars before heading to the woods#2019-07-2319:38geraldodevToday I've found https://clojars.org/luchiniatwork/cambada which has been forked by https://github.com/mikeananev/cambada . Are you using any of those ? How are you doing aot in tools-deps. The fork is 1.0.4 , they are realeasing more often but the issue list is disabled.#2019-07-2320:30aviI’m using Cambada in my project to do AOT and create an überjar. It’s been working well for me.#2019-07-2321:28sogaiusome things i've heard of include: https://github.com/EwenG/badigeon https://github.com/juxt/pack.alpha#2019-07-2323:04hiredmanthe clj script I have (version 1.9.0.375 I guess? from the -Sdescribe, not sure how to track that to a tools.dep version) doesn't seem to handle '-i' in what I thought was the correct way. my understanding is -i means "load this file and give me a repl", and what happens is it loads the file and then exits#2019-07-2323:07seancorfieldYou need -r to get a REPL if you specify any of the main opts I think.#2019-07-2323:07seancorfield(and it's still true on the latest version of the clj script I'm fairly sure)#2019-07-2323:08seancorfieldConfirmed: (! 693)-> clj -Sdescribe
{:version "1.10.1.466"
...
Tue Jul 23 16:08:19
(sean)-(jobs:0)-(~/clojure)
(! 694)-> clj -i script.clj
Hello from script
Tue Jul 23 16:08:29
(sean)-(jobs:0)-(~/clojure)
(! 695)-> clj -i script.clj -r
Clojure 1.10.1
Hello from script
user=> #2019-07-2323:09hiredmanah, I wonder if that has always been the case and I just forgot about it because of how little I use -i#2019-07-2323:10hiredmanlooks like it#2019-07-2323:22alexmillerThat has been the same forever #2019-07-2419:21Travisjust curious but is it possible to pull dependencies from a Google Cloud Storage based maven repo ?#2019-07-2419:27alexmillerif it acts as a Maven repo on http, then sure#2019-07-2419:28alexmillerif not, then no#2019-07-2419:28alexmillerhow do you configure it in Maven?#2019-07-2419:29alexmillerif there's a custom wagon, then that will not work#2019-07-2419:30alexmillerbut it could be made to do so#2019-07-2419:34Travisneed to research more. Currently I have a jar that is being built/published by gradle into GCS using there maven-publish plugin. Basically to avoid managing some private artifact repository and just use GCS#2019-07-2419:45alexmillersure, this is commonly done with s3 (and tools.deps includes support for that)#2019-07-2421:50seancorfieldI'm modifying depstar to insert a MANIFEST.MF file and pom.xml / pom.properties. My minimalist thinking is that it should do this automatically if it finds a pom.xml file in the directory in which it is run -- as part of the project you are building a JAR/uberjar for. Is this a good idea and if not, why not?#2019-07-2421:52seancorfieldI'm thinking you'd do clj -Spom && clj -A:jar my-project.jar (assuming the :jar alias brings in depstar and uses -m hf.depstar.jar).#2019-07-2421:53seancorfieldAre there situations where automatically picking up the local pom.xml file is a bad idea? Are there situations where you would want to use a different file as the POM or you might have the POM in a different location?#2019-07-2421:55dorabNot directly answering your question, but any changes to depstar that allow me to java -jar foo.jar rather than java -cp foo.jar clojure.main -m foo.core would be useful.#2019-07-2422:00seancorfield@dorab that would require depstar to deal with AOT -- not happening.#2019-07-2422:01seancorfieldWhat this change will allow for is java -jar foo.jar -m your.entry.point instead of java -cp foo.jar clojure.main -m your.entry.point#2019-07-2422:03seancorfield(assuming you had pom.xml locally -- so depstar would generate a manifest and because you're generating an uberjar, instead of a library JAR, you'd get Main-Class: clojure.main added)#2019-07-2422:06seancorfieldI guess my real question is whether it would be useful/important to add a flag to suppress consumption of pom.xml?#2019-07-2422:15seancorfield(easy enough to do, so I might as well)#2019-07-2422:31alexmillerJava doesn’t care about the pom stuff so that seems harmless. clj will use it to find dependencies though if you use it as a local file dep#2019-07-2422:31alexmillerThat’s prob good if not an uberjar and bad if it is an uberjar#2019-07-2422:33alexmiller(As you’d get deps from two places on the classpath)#2019-07-2422:40seancorfieldYeah, that's why this has cropped up -- http://cljdoc.org assumes the JAR contains it.#2019-07-2422:40seancorfieldAnd with that ... seancorfield/depstar "0.3.0" is available!#2019-07-2423:44dorab@seancorfield java -jar foo.jar -m your.entry.point is good enough for me. Thanks for the release. Will try it out.#2019-07-2500:41ghadi@seancorfield is the main class automatically inserted into the jar now? (Haven't tried it)#2019-07-2500:42seancorfieldIf you build an uberjar, it adds Main-Class: clojure.main#2019-07-2500:43seancorfieldSince it does no AOT, you still have to go through clojure.main.#2019-07-2816:20borkdudelocal/root is only used for development on a local machine? e.g. you cannot specify a local/root dep in a git dep?
example:
project/examples/deps.edn <- used as git dep, refers to library on level up as :local/root ".."
project/deps.edn <- library#2019-07-2817:43alexmillerWith the most recent set of dir related fixes, I think that should work#2019-07-2817:43alexmillerDid you try it?#2019-07-2818:15borkdudeI did try it, let me see if I have the latest#2019-07-2818:25borkdudeupgraded and I can confirm this works. this is awesome, thanks#2019-07-2818:25alexmillerCool#2019-07-2818:34borkdudewhat's cool is that it now works in dev but also as a git dep, without having to change anything in deps.edn:
https://github.com/borkdude/clj-kondo/blob/dump-analysis/tools/deps.edn#2019-07-2818:43dominicmthere was another set of changes...?#2019-07-2818:45dominicmhttps://clojure.atlassian.net/browse/TDEPS-132 is this resolved then?#2019-07-2818:45borkdudeit seems so 🙂#2019-07-2818:46borkdudethe version I'm using right now is 1.10.1.466#2019-07-2818:52dominicmNothing in the changelog indicates tdeps-132 is resolved. I think this is still an issue.#2019-07-2818:53dominicmEssentially if you have 2 git dependencies in that repo, that both have a :local/root on C, then you'll hit an error: https://clojure.atlassian.net/browse/TDEPS-123
This is important for people who want to use Edge without cloning it.#2019-07-2818:53borkdude:local/root on C?#2019-07-2818:54dominicmgiven deps, A, B, C:
deps.edn
- A {:git/url ...}
- C {:local/root "../C"}
- B {:git/url ...}
- C {:local/root "../C"}
#2019-07-2818:55borkdudeah#2019-07-2821:31carkhhum cannot access this link if logged in service desk#2019-07-2821:32carkhworks when logged out though#2019-07-2821:32carkh(not a deps issue)#2019-07-2910:59borkdude@dominicm good to know that doesn't work yet - if it did, that would be a reason for me to jump from boot to tools.deps in our local codebase, since we have locally shared deps now that we must deploy somewhere, which feels a bit icky#2019-07-2914:13alexmilleras TDEPS-132 indicates, I consider this out of scope for now, no plans to work on that#2019-07-2917:57borkdudeis there something in tools.deps which combines R and C? so I want all test deps but also the test dir to be on the classpath#2019-07-2917:57borkdudeso clojure -Rtest -Ctest -Spath#2019-07-2917:58alexmiller-A#2019-07-2918:05borkdudebut won't that also execute a main?#2019-07-2918:14alexmilleryes, if there is one in the test alias. you didn't mention that#2019-07-2918:15alexmillerif that's the case, then I'd say no, use -R and -C#2019-07-2918:16borkdudewell, I only know after looking, so it's not something I would use by default that way#2019-07-2918:16borkdudeand people might add one later, so if I do that in a script, the script might break#2019-07-2918:16borkdudeso -R and -C is fine#2019-07-2923:05kennyEvery time I run clj in some of my projects, I get all of this outputted:
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-netty-shaded/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Is this necessary?#2019-07-2923:12seancorfield@kenny That indicates you are either depending on SNAPSHOT releases or RELEASE/LATEST rather than specific non-snapshot versions.#2019-07-2923:13kennyThat would surprise me. Would that show up in -Stree?#2019-07-2923:13seancorfieldSince snapshots and both RELEASE/LATEST can refer to different versions over time, the dependency checker must fetch metadata to check which version is current.#2019-07-2923:13seancorfieldYeah, I would expect it to show up in -Stree#2019-07-2923:14kennyNothing shows up using any of those identifiers.#2019-07-2923:15seancorfieldWell, something is trying to pull in io.grpc/grpc-core and/or io.grpc/grpc-api...#2019-07-2923:16kennyCorrect but -Stree shows the version locked.#2019-07-2923:16seancorfield-Stree shows what version it is resolved to -- not necessarily how it was declared.#2019-07-2923:17seancorfieldSo using the output of -Stree you can figure out what path through the dependencies was taken -- and then look in your deps.edn to see how you're pulling those top-level things in...#2019-07-2923:17kennyWell it's from this dep: com.google.cloud/google-cloud-monitoring {:mvn/version "1.78.0"}#2019-07-2923:19kennyI think it'd be weird for that lib to use one of those identifiers.#2019-07-2923:19kennyI don't see it here: https://repo1.maven.org/maven2/com/google/cloud/google-cloud-monitoring/1.78.0/google-cloud-monitoring-1.78.0.pom#2019-07-2923:19seancorfieldThat specifies dependencies with no versions https://search.maven.org/artifact/com.google.cloud/google-cloud-monitoring/1.78.0/jar#2019-07-2923:20kennyAh, was just ctrl+f'ing. That's a weird way to do it.#2019-07-2923:20kennySo because they didn't specify a version, tools-deps tries to find the latest one each time?#2019-07-2923:20seancorfieldYup.#2019-07-2923:21kennyBleh. Why would they do that?#2019-07-2923:21seancorfieldBecause they're Google? You can probably fix that by putting those dependencies explicitly in your project with appropriate versions...#2019-07-2923:22seancorfieldAh, looks like they specify them in the parent pom https://search.maven.org/artifact/com.google.cloud/google-cloud-clients/0.96.0-alpha/pom#2019-07-2923:23seancorfieldSo maybe tools.deps doesn't follow that path? @alexmiller might be able to speak to that.#2019-07-2923:23seancorfieldBut at least that gives the specific versions you'd be expected to use (with GCM 1.78.0)#2019-07-2923:24kennyIs a parent supposed to be included in the dep tree or only used to resolve artifacts in the child?#2019-07-2923:24seancorfield(look at the <properties> section of that parent pom I linked to)#2019-07-2923:25seancorfieldThat question is beyond my Maven knowledge, sorry.#2019-07-2923:26kennySo weird. Everything works as expected in the REPL.#2019-07-2923:26seancorfieldin the clj repl?#2019-07-2923:26kennyYep#2019-07-2923:27seancorfieldRight, because it resolves all the versions.#2019-07-2923:28seancorfieldI'm just suggesting you can suppress those metadata fetches by explicitly declaring dependencies in deps.edn on the relevant versions of io.grpc/grpc-core, io.grpc/grpc-api, etc.#2019-07-2923:28seancorfieldI have a bunch of dev-only tooling where I use RELEASE as the version and every day clj will show messages about fetching metadata -- to check the most recent version.#2019-07-2923:29seancorfieldIn your -Stree output, what version of those io.grpc libs does it show that it resolves to?#2019-07-2923:29kenny1.21.0#2019-07-2923:31seancorfieldMakes sense. The latest GCM's parent pom still uses that version too. https://search.maven.org/artifact/com.google.cloud/google-cloud-clients/0.102.0-alpha/pom#2019-07-2923:33kennyAdding io.grpc/grpc-core {:mvn/version "1.21.0"} fixes this.#2019-07-2923:33seancorfieldYup. That's what I'd expect.#2019-07-3102:45minikomiwell, it took a little while, but i'm super sold on deps.edn
i had to write my own watch/reload logic for parts i was working on.. but it was actually fairly painless using ns-tracker
love the ease of switching between maven/git/local deps when developing libraries .. really encourages breaking useful parts out sooner rather than later#2019-07-3102:45minikomii tend to use mount a lot more too to get my garden/cljs/web server/nrepl scaffolding up and running & reloadable#2019-07-3102:47carkhyes the local deps are so helpful for exercising an ongoing library project#2019-07-3102:48minikomidocs for some things like using figwheel as part of a project rather than straight from the command line are a bit thin, but i fell much more in tune with my stack now that i've ventured deeper down and built out the parts i need haha#2019-08-0313:19sekaoam i right that there is no way to combine aliases in deps.edn? so instead of writing clj -A:one:two i could just write clj -A:two and it would pull in :one automatically. i know you can't pass aliases to :main-opts so i'm guessing i just need to write them both explicitly#2019-08-0314:22alexmillercorrect#2019-08-0320:39dharriganAny pointers to approaches/libraries that are similar to the intent of pomegranate/alembic to allow for reloading of deps.edn within a running repl (to pull in a new dep) or to dynamically add in a dep on the repl (to kickstart a download and adding to the classpath)?#2019-08-0320:49seancorfield@dharrigan tools.deps has a branch with add-lib that dynamically adds new libraries to the running system...#2019-08-0320:49dharrigansweet, ta @seancorfield!#2019-08-0320:50dharriganI guess my next question is (when) will it be merged in and released?#2019-08-0320:50seancorfieldYou can use it from GitHub today https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L106-L119#2019-08-0320:50dharrigankk#2019-08-0320:51dharriganpretty awesome 🙂#2019-08-0320:51seancorfieldThat also has an example of how to add master from GitHub 🙂#2019-08-0320:51dharriganthanks 🙂#2019-08-0519:52seancorfieldseancorfield/depstar {:mvn/version "0.3.1"} was just released -- includes a new -m / --main option so that if you have a pom.xml and do your own AOT/compile step to create a "main" entry point, you can now override the Main-Class in the generated manifest (overriding the default of clojure.main).#2019-08-0519:56ghadidepstar @seancorfield?#2019-08-0519:56ghadithe coord might be missing a piece#2019-08-0520:11seancorfieldUpdated! Thanks!#2019-08-0612:24dharriganSo#2019-08-0612:25dharriganPulling in tools.deps.alpha via a git/url hash, to access the branch that allows me to load in a jar from the repl#2019-08-0612:25dharriganI'm also using slf4j#2019-08-0612:25dharriganMy logs stopped working (i.e., output)#2019-08-0612:26dharriganSLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/david/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/david/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
#2019-08-0612:26dharriganaccording to the doc, the choice is totally random#2019-08-0612:26dharriganwhy does deps include the nop static binding?#2019-08-0612:26dharriganI can't exclude it, like I can if I'm doing "mvn/version"#2019-08-0612:27dharrigan(using :exclusions)#2019-08-0612:27dharriganorg.clojure/tools.deps.alpha d0b33e0
...
...
chopped for brevity
...
org.slf4j/slf4j-nop 1.6.2
#2019-08-0612:28dharriganFrom the slf4j doc#2019-08-0612:28dharriganThe warning emitted by SLF4J is just that, a warning. Even when multiple bindings are present, SLF4J will pick one logging framework/implementation and bind with it. The way SLF4J picks a binding is determined by the JVM and for all practical purposes should be considered random. As of version 1.6.6, SLF4J will name the framework/implementation class it is actually bound to.
#2019-08-0612:29alexmillerI think we have a ticket for this, probably need to stop including that#2019-08-0612:30dharriganYes please! Having a library link in the static binding ain't right 🙂#2019-08-0612:30alexmillerWhy did the exclusion not work?#2019-08-0612:30dharrigannot supported, according to the docs:#2019-08-0612:30dharriganhttps://clojure.org/reference/deps_and_cli#_dependencies#2019-08-0612:31dharriganyou can for maven coordinates, but not for git coordinates#2019-08-0612:31alexmillerDid you try it?#2019-08-0612:31dharriganYup#2019-08-0612:31dharriganorg.clojure/tools.deps.alpha {:git/url ""
:sha "d0b33e0d346736aa985c150145b332f97b92135e"
:exclusions ["org.slf4j/jcl-over-slf4j"
"org.slf4j/slf4j-nop"] }}}
#2019-08-0612:31alexmillerShould be a symbol, not a string#2019-08-0612:32alexmillerI think it will work#2019-08-0612:32dharriganah good spot#2019-08-0612:32dharriganretrying#2019-08-0612:33dharriganthat worked, after rm -rf'ing my .cpcache dir#2019-08-0612:34dharriganta very much, but still, I think the nop should be excluded totally, may catch others out 😉#2019-08-0612:34alexmillerYeah#2019-08-0612:34dharriganparrot#2019-08-0613:15rickmoynihanHey a new colleague is getting the following error when resolving a git dep in a private github repo. Not sure what to do… I’ve seen the ticket https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-91 which looks relevant.#2019-08-0613:15rickmoynihan#2019-08-0613:17rickmoynihanHer ssh key appears to be the same version as mine which works…
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,xxxx
#2019-08-0613:18rickmoynihanshe’s also running ssh-agent and has added the key#2019-08-0613:44alexmillerso ssh-add -l lists her identity?#2019-08-0613:45alexmillerand I assume she does actually have access to the git repo (can access via git)#2019-08-0613:48alexmillerI've found on mac that I had a setting in my ~/.ssh/config under "Host *" that had a setting for IdentityFile or something like that which I had to delete#2019-08-0613:48rickmoynihanyes to both of those#2019-08-0613:48rickmoynihan👀#2019-08-0613:49alexmillermy ssh config looks like:#2019-08-0613:49alexmillerHost *
AddKeysToAgent yes
UseKeychain yes
#2019-08-0613:49rickmoynihanthanks @alexmiller dropping IdentityFile works#2019-08-0613:49alexmillerssh ninja#2019-08-0613:50rickmoynihan:punch:#2019-08-0613:51alexmillerI added that to that jira issue in a comment#2019-08-0613:52rickmoynihan👍 my future self with thank you again for that#2019-08-0613:56alexmillerditto, I had to look it up (again)#2019-08-0614:21rgm@alexmiller 🎉 … private deps seem to be working for me now#2019-08-0614:25rgm(to be clear, this is just a general man-are-there-ever-a-lot-of-moving-parts-to-auth gripe, not a tools.deps gripe 🙂)#2019-08-0618:24zaneI'm curious how y'all are switching between different Java versions when using clj and clojure.#2019-08-0618:25alexmillerhint: the answer has nothing to do with clj and clojure :)#2019-08-0618:25alexmillerthey just use whatever java is on the path#2019-08-0618:26alexmillerso to rephrase, how are y'all switching between Java versions on your OS of choice?#2019-08-0618:28zane> they just use whatever java is on the path
Got it. So that means that clj and clojure will ignore the version set by, for example, jenv?#2019-08-0618:30zaneI suppose since jenv works by providing a java shim it ought to still work provided that the jenv shim is the first java on the path. 🤔#2019-08-0618:31alexmillerI don't know what jenv does but that seems right#2019-08-0618:40zaneThank you! 🙂#2019-08-0619:17seancorfield@zane I set JAVA_HOME to point to the JDK installation that I want to use... (! 586)-> JAVA_HOME=$OPENJDK11_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"11.0.1"
user=> ^D
Tue Aug 06 12:16:46
(sean)-(jobs:0)-(~/clojure)
(! 587)-> JAVA_HOME=$OPENJDK8_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"1.8.0_192"
user=>
#2019-08-0619:17seancorfield@zane I set JAVA_HOME to point to the JDK installation that I want to use... (! 586)-> JAVA_HOME=$OPENJDK11_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"11.0.1"
user=> ^D
Tue Aug 06 12:16:46
(sean)-(jobs:0)-(~/clojure)
(! 587)-> JAVA_HOME=$OPENJDK8_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"1.8.0_192"
user=>
#2019-08-0621:26zaneHey Sean, thanks for sharing your process! This only changes the JAVA_HOME for the command that follows, right? (Or for the shell that does the export if export is included.)#2019-08-0621:27seancorfieldJust for that command, yes. Which is why it's nice for testing.#2019-08-0621:29zaneMakes a ton of sense for testing. I've done that myself.#2019-08-0621:30zaneThe particular use case I'm solving for is when I have a long-running Emacs process that occasionally launches clojure processes via cider-jack-in.#2019-08-0621:30zaneIn that case setting JAVA_HOME in the shell won't result in it getting picked up by the already-running Emacs process.#2019-08-0622:47seancorfieldBut you could get Emacs to set the env var for the invocation of clojure by changing the command line it uses?#2019-08-0622:59zaneIf I'm understanding you correctly, probably? Would require delving deeper into Cider than I want to do right now. 🙂#2019-08-0714:04Ahmed Hassan@zane you can connect to running clojure process by using cider-connect-clj#2019-08-0714:05Ahmed HassanSimilarly for ClojureScript you can use cider-connect-cljs#2019-08-0714:15zaneYes, I'm aware. I prefer to let Cider launch Clojure because choosing compatible versions of all the middleware can be difficult. But thank you! 🙂#2019-08-0716:04seancorfield"because choosing compatible versions of all the middleware can be difficult" -- a big part of why I stopped using Emacs/CIDER/nREPL #2019-08-0716:11Ahmed Hassan@U04V70XH6 What do you use now?#2019-08-0716:11seancorfieldI switched from Emacs to Atom/ProtoREPL and then to Atom/Chlorine. I can connect Atom/Chlorine to any Clojure process since it only requires a Socket REPL.#2019-08-0716:12seancorfield(so I can, and do, connect Atom/Chlorine to QA and production processes that were configured to start a Socket REPL using JVM options -- no dependencies needed in the project, no logic to start a custom REPL)#2019-08-0716:13Ahmed HassanHow is experience of Atom/Chlorine as compared with Cider/Emacs?#2019-08-0716:17seancorfieldI find it a lot smoother and a lot less "fiddly". I talked a bit about why I switched from Emacs in this blog post https://corfield.org/blog/2018/12/19/atom-chlorine/#2019-08-0717:19zaneYeah. I can understand why people make that switch, though for me it wouldn't be worth it.#2019-08-0620:01dominicmI think the way voidlinux does it is neat. JAVA_HOME points at a symlink, and xbps-alternatives changes the symlink.#2019-08-0621:08dharriganI use arch, so I do sudo archlinux-java set java-12-openjdk#2019-08-0621:08dharriganor sudo archlinux-java set java-8-openjdk#2019-08-0621:08dharriganeasy peasy#2019-08-0701:12cflemingIf I want to use a fork of Clojure with deps, can I exclude the clojure version pulled in from the system deps.edn somehow?#2019-08-0704:13seancorfield@cfleming If your local deps.edn file has that (local?) Clojure in :deps directly, it should override the system deps anyway?#2019-08-0705:19alexmillercorrect ^^#2019-08-0705:19alexmillerassuming you have built and installed that fork locally in Maven so that it's artifact id is still org.clojure/clojure#2019-08-0705:20alexmillerif you want to call it something else (or use as a git or local dep), then no, that is not currently supported#2019-08-0705:22alexmillerthat's not really intentional just kind of a consequence of various assumptions#2019-08-0705:23cflemingYeah, I changed the artifact ID to reduce confusion.#2019-08-0705:23cflemingHmm#2019-08-0705:23alexmillertop-level exclusions would possibly help with this but not there yet#2019-08-0705:23cflemingOk, I think it would be kind of a pain to use due to… right, the lack of that.#2019-08-0705:24cflemingI’ll stick with Maven for now then, thanks!#2019-08-0705:25cflemingActually, if I had the forked jar in a lib dir, and used :deps {org.clojure/clojure {:local/root "lib/clojure-1.10.1.jar"}}, that should work, right?#2019-08-0705:25cflemingi.e. deps will assume that the artifact ID is whatever I tell it.#2019-08-0705:26cflemingAnd that will override the official one.#2019-08-0705:35alexmillerthere are some issues with overriding a maven dep with a git / local dep#2019-08-0705:35alexmillerso I don't think it will work#2019-08-0705:36cflemingOk, thanks. I’ll ditch that idea for now, then.#2019-08-0705:36alexmillerthat one kind of bugs me. I haven't had a chance to look at it yet#2019-08-0705:36alexmillerpart of it is the version comparison across dep types#2019-08-0705:37alexmillerbut on override that shouldn't matter#2019-08-0720:56hkuptyHi folks, has anyone successfully used tools.deps w/ git repos on private repositories?#2019-08-0720:59alexmillerYes#2019-08-0721:12hkuptyAs the maintainer of both tools.deps and tools.gitlibs, I expected a more user-friendly answer from you.#2019-08-0721:13alexmillerSorry, was just answering your question. Was hard to provide any additional detail without any problem description#2019-08-0721:02hkuptyI'm having issues w/ unknown host key if I use {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "86e1eff2c6a8a8a8"}, :content ("[email protected]")} or I need to provide credentials if I use the https form, which I obviously won't do, since I'm not comiting credentials on the repo#2019-08-0721:02hkuptyIs there any docs on how to get over that?#2019-08-0721:13alexmillercan you share the error message or more info about your setup and what you're doing?#2019-08-0721:13alexmillerwhat os are you on?#2019-08-0721:14alexmillerhave you been through the git stuff at https://clojure.org/reference/deps_and_cli#_procurers ?#2019-08-0721:15alexmillerhttps auth is not currently supported so you'll need to use ssh#2019-08-0721:16hkuptyI'm running from Fedora 30, very basic setup.
The error code I get is:
Execution error (JSchException) at com.jcraft.jsch.Session/checkHost (Session.java:805).\nUnknownHostKey: . RSA key fingerprint is .....
It came from a direct invocation of tools.deps respolve-deps:
(deps/resolve-deps {:deps {'project {:git/url "#2019-08-0721:17hkuptyI'm running the repl from a session where I've added the keys to ssh-agent, so I assume this step is done.#2019-08-0721:18alexmillerlooks like same as https://clojure.atlassian.net/browse/TDEPS-104#2019-08-0721:18carkhdid you add the host to your .ssh/known_hosts ?#2019-08-0721:18alexmillerthere's a workaround described at https://codehopper.nl/2018/09/27/clojure-gitlab-deps/#2019-08-0721:18hkuptyHum... I think I know what could be the issue.#2019-08-0721:19hkuptyI'm using ED25519 instead of RSA for gitlab.#2019-08-0721:19ghadissh-keyscan -t rsa,ed25519 >> ~/.ssh/known_hosts#2019-08-0721:20ghadithat will seed UnknownHostKeys ^ into your config#2019-08-0721:21ghadiED25519 on the user identity file is fine (when using an agent) @ingvij
the error you're pasting is an rsa host key, a different kind of key#2019-08-0721:22hkuptyI see. Fine then, I'll try both approaches and I'll post the results back here.#2019-08-0721:22ghadiboth?#2019-08-0721:23hkuptyssh-keyscan and the link @alexmiller posted#2019-08-0721:23ghadithe article @alexmiller posted seems to use the same exact thing I'm describing#2019-08-0721:23hkuptyahh, sorry. I hadn't read the article yet#2019-08-0721:23ghadisince clojure deps.edn is not terminal interactive, the .ssh/known_hosts file needs the host key#2019-08-0721:24hkuptyRight, that makes sense.#2019-08-0721:25ghadi(but this approach is a bit insecure because no one is validating the fingerprints...)#2019-08-0721:25ghadi(then again most people type Yes anyways when ssh prompts you..)#2019-08-0721:27hkuptyShould work fine as a workaround I suppose.#2019-08-0721:29hkuptyThat worked for checking the dependency, but I'm getting a MetadataNotFoundException now. I'll double-check everything, specially the dependency itself. Thanks for the help @ghadi and @alexmiller.#2019-08-0721:31alexmillerif you can share more info, might have some idea what that is, but I don't even know what layer of the stack it's from#2019-08-0721:32alexmillerjust in general, providing information at the beginning lets people help you more efficiently, and we're all doing this for free, in spare time#2019-08-0721:32hkuptyExecution error (MetadataNotFoundException) at org.eclipse.aether.internal.impl.DefaultMetadataResolver/resolve (DefaultMetadataResolver.java:233).\nCould not find metadata org.clojure:clojure/maven-metadata.xml in local (/home/hkupty/.m2/repository)
#2019-08-0721:36hkuptyThe stacktrace from tools.deps:
[[org.eclipse.aether.internal.impl.DefaultMetadataResolver
resolve
"DefaultMetadataResolver.java"
233]
[org.eclipse.aether.internal.impl.DefaultMetadataResolver
resolveMetadata
"DefaultMetadataResolver.java"
191]
[org.apache.maven.repository.internal.DefaultVersionResolver
resolveVersion
"DefaultVersionResolver.java"
233]
[org.eclipse.aether.internal.impl.DefaultRepositorySystem
resolveVersion
"DefaultRepositorySystem.java"
238]
[clojure.tools.deps.alpha.extensions.maven$eval362$fn__365
invoke
"maven.clj"
40]
[clojure.lang.MultiFn invoke "MultiFn.java" 239]
[clojure.tools.deps.alpha$canonicalize_deps$fn__761
invoke
"alpha.clj"
68]
[clojure.core.protocols$fn__8159 invokeStatic "protocols.clj" 168]
[clojure.core.protocols$fn__8159 invoke "protocols.clj" 124]
[clojure.core.protocols$fn__8114$G__8109__8123
invoke
"protocols.clj"
19]
[clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 31]
[clojure.core.protocols$fn__8144 invokeStatic "protocols.clj" 75]
[clojure.core.protocols$fn__8144 invoke "protocols.clj" 75]
[clojure.core.protocols$fn__8088$G__8083__8101
invoke
"protocols.clj"
13]
[clojure.core$reduce invokeStatic "core.clj" 6828]
[clojure.core$reduce invoke "core.clj" 6810]
[clojure.tools.deps.alpha$canonicalize_deps
invokeStatic
"alpha.clj"
67]
[clojure.tools.deps.alpha$canonicalize_deps invoke "alpha.clj" 65]
[clojure.tools.deps.alpha$expand_deps$fn__787
invoke
"alpha.clj"
184]
[clojure.tools.deps.alpha$expand_deps invokeStatic "alpha.clj" 183]
[clojure.tools.deps.alpha$expand_deps invoke "alpha.clj" 164]
[clojure.tools.deps.alpha$resolve_deps invokeStatic "alpha.clj" 231]
[clojure.tools.deps.alpha$resolve_deps invoke "alpha.clj" 213]
#2019-08-0721:38alexmillerorg.clojure:clojure/maven-metadata.xml looks weird#2019-08-0721:38alexmillermaybe bad format in your deps.edn?#2019-08-0721:39alexmillerorg.clojure:clojure when should be org.clojure/clojure maybe?#2019-08-0721:42hkuptyI checked the dependency, it's clojure version was RELEASE as it comes from default when you bootstrap a project w/ https://github.com/seancorfield/clj-new#2019-08-0721:43ghadiboo#2019-08-0721:44ghadiyou should choose an explicit version for clojure#2019-08-0721:45ghadi@alexmiller maybe -Sresolve can rewrite symbolic maven things, too#2019-08-0721:46hkuptyI know. this is still on very initial stages, just testing really..#2019-08-0721:46hkuptyThat's why I didn't care too much about it. Just pinned the version.#2019-08-0721:47alexmillerre RELEASE, agreed, bleh#2019-08-0721:48alexmillerRELEASE is not officially supported in deps.edn#2019-08-0721:48alexmillerI've let it continue to work as it's sometimes a good hack, but shouldn't be enshrining it#2019-08-0721:50alexmiller-Sresolve is going to go away (move into an external tool)#2019-08-0721:53hkuptyI moved past that error, now yet another one bothering me. A dependency of a dependency is not being resolved, whereas if I run clj on the library I'm trying to import, it works fine:
Syntax error (ExceptionInfo) compiling at (cli/ops.clj:5:1).\nUnable to resolve io.grpc/grpc-core version: [1.17.1]
When I run the repl directly it works, if I call this it fails:
(deps/resolve-deps {:deps {'io.etcd/jetcd-core {:mvn/version "0.3.0"}}}
{})
#2019-08-0721:55hkuptyI don't want to keep you involved in the debugging unless you want to, I just keep posting since I think those might be relevant. I'd understand if you'd rather want me to open an issue.#2019-08-0721:56hkuptywhich, btw, I'll do for clj-new so new projects are not created w/ RELEASE on clojure version.#2019-08-0721:57hiredmanthat is a syntax error, not related to deps at all#2019-08-0721:57hiredmanI would dig in to cli/ops.clj on the given line number, likely something isn't being quoted correctly#2019-08-0722:00hkuptythat particular invocation is this one:
(deps/resolve-deps {:deps {'io.etcd/jetcd-core {:mvn/version "0.3.0"}}}
{})
I believe this might be due to some parsing from pom.xml instead of deps.edn maybe#2019-08-0722:02hiredmanon pardon me, that does come from tools deps, I wonder why it is categorized as a syntax error#2019-08-0722:04hkuptyI assume it's complaining about the dependency declaration syntax, as it expects ns:symbol:jar:version where version is a string, but instead it gets a list.#2019-08-0722:05alexmiller@U0NCTKEV8 the syntax thing is a bug, I have a patch pending for that#2019-08-0722:05hiredmanclj -Sdeps "{:deps {io.etcd/jetcd-core {:mvn/version \"0.3.0\"}}}" might give you a clearer error#2019-08-0722:06hiredmanit looks like something depends on an artifact that is not in maven central#2019-08-0722:06alexmillerif you're calling this with tools.deps directly, you need to pass it the repos#2019-08-0722:06alexmillerclj provides those, but tools.deps does not (automatically)#2019-08-0722:07alexmillerthere's an example at https://github.com/clojure/tools.deps.alpha/blob/master/API.md#2019-08-0722:08hkuptyI'll check that. However, the thing is I can do
(deps/resolve-deps {:deps {'io.grpc/grpc-core {:mvn/version "1.17.1"}}} {})#2019-08-0722:09hiredmanbut that is a different thing#2019-08-0722:09alexmillerb/c you have it locally#2019-08-0722:09hkuptyBut I can't do the same for io.etcd/jetcd-core, which fails on this very same dependency.#2019-08-0722:11hiredmanit does not for me, for me it fails to find io.netty:netty-codec-http2:jar:[4.1.30.Final] in central#2019-08-0722:12hkuptyRight. I got a Syntax error (ExceptionInfo) compiling at (cli/ops.clj:5:1).\nCould not find artifact io.grpc:grpc-core:jar:1.17.1 after moving my ~/.m2 to ~/.m2-bkp`.#2019-08-0722:12hkuptySo I'll make sure to add the extra repositories#2019-08-0722:13hiredmanit would be great if you could try the command line call to clj#2019-08-0722:14hiredmanit looks like the place in deps where that exception is thrown is if the dep is either a version range or an alias like LATEST or RELEASE#2019-08-0722:14hiredmanso my guess something in jetcd is changing how maven attempts to resolve an alias or a range in io.grpc's dependencies#2019-08-0722:15hkuptyI could make it work by adding "central" {...} to :mvn/repos#2019-08-0722:15hkuptythat solves everything#2019-08-0722:16hiredmanhrmm, I am still getting errors, and it has changed to different artifacts not being found in central now 😞#2019-08-0722:16hkupty😱#2019-08-0722:16hiredmanbut I am not using jetcd-core so no big deal#2019-08-0722:18hkupty😅 Thanks to you both for all the help and guidance 🙂#2019-08-0722:28seancorfieldIn light of continued complaints about the use of "RELEASE" in projects created by clj-new ... seancorfield/clj-new {:mvn/version "0.7.7"} pins these to 1.10.1 (for Clojure) and 0.10.0-RC1 (for test.check). You're welcome 🙂#2019-08-0722:29seancorfieldBear in mind that lein new, boot new, and clj-new all look for templates themselves (on Clojars/Central) using "RELEASE" under the hood unless you specify a template version to use (which almost no one does).#2019-08-0722:30ghadithanks @seancorfield#2019-08-0723:14andy.fingerhutThis is not a critical question in any way, but was curious whether folks had any recommendations on creating a deps.edn file for a library that has Clojure-specific and ClojureScript-specific flavors in the same git repo, e.g. core.rrb-vector is the one I have in mind. One thought: create a deps.edn that has almost 0 dependencies when no aliases are specified, and has a :clj alias for Clojure dependencies/paths, and a :cljs alias for ClojureScript dependencies/paths. Second thought: Two separate deps.edn files, except there is only one root directory file named deps.edn available.#2019-08-0723:15andy.fingerhutThird thought: two separate git repos, one for each variant?#2019-08-0802:08alexmillerJust fyi on kind of a tangent, core.rrb-vector is a Clojure contrib and the only build officially supported for those for build and deploy is the pom.xml - that has to be the primary#2019-08-0802:09andy.fingerhutUnderstood.#2019-08-0802:09alexmillerit's ok for it to also have a deps.edn#2019-08-0802:09alexmillerand the tool for variant classpaths is aliases#2019-08-0802:09andy.fingerhutBut the pom.xml is a no-op for ClojureScript, isn't it?#2019-08-0802:10andy.fingerhutoh, maybe not.#2019-08-0802:10alexmillerwe have other contrib libs that build a single jar usable from either#2019-08-0802:10alexmillerdata.xml, tools.reader, etc#2019-08-0802:10seancorfieldtools.cli https://github.com/clojure/tools.cli/blob/master/deps.edn#2019-08-0802:10andy.fingerhutI guess the pom.xml controls what ClojureScript source files are bundled up in there.#2019-08-0802:10alexmilleryes#2019-08-0802:10seancorfield(tools.cli is pure .cljc so it may not be a good example for you)#2019-08-0802:11alexmillerthe trickiest part of clj/cljs contribs is testing and probably data.xml is the most advanced example out there#2019-08-0802:14andy.fingerhutI will take a look at data.xml more carefully. I know pretty much all contribs do automated build and at least a subset of their tests automated with results on http://build.clojure.org. Do you know if any of them build and test ClojureScript code?#2019-08-0802:17andy.fingerhutAh, I see in the console log for data.xml that it does run some cljs tests using nashorn JavaScript runtime.#2019-08-0802:20seancorfieldI should have a look at that too since the Maven build for tools.cli only does clj testing, and I do cljs testing manually via deps.edn...#2019-08-0802:22alexmillerthose are the main ones I can think of that do cljs testing#2019-08-0802:23alexmiller"building" is really just including the cljs in the jar so not really a big deal, a bunch of contribs do that#2019-08-0802:23alexmillerin a lib like core.async there is a manual cljs test process that we run by hand, but it's not part of ci#2019-08-0802:26andy.fingerhutSo that is a great lead-in to what the pom.xml should ideally do (i.e. automate ClojureScript as well as Clojure tests), and I can look into that. Definitely nicer with a working example to start from.#2019-08-0802:26andy.fingerhutAs far as deps.edn file goes, would it be bad to have one, if its contents conflict with what pom.xml says?#2019-08-0802:27alexmillerno, not bad#2019-08-0802:28alexmillermany of the contribs also have project.clj files#2019-08-0802:29andy.fingerhutdeps.edn in a Maven-deployed JAR file should either (a) not be in the JAR, or (b) ignored by tools.deps, Maven, etc. even if one is in there?#2019-08-0802:31alexmillerI don't think it would be in the jar file#2019-08-0802:31alexmillerand it would be ignored by Maven#2019-08-0802:32alexmillerthere are plenty of examples to check -a bunch have deps.edn files#2019-08-0802:32alexmillertools.deps won't ignore it but presumably that's the point#2019-08-0802:32alexmillerif this gets too into the weeds on contrib, I would move it to #clojure-dev#2019-08-0802:33andy.fingerhutUnderstood. If I haven't exhausted you with questions yet, is it short to explain what happens when tools.deps follows a dependency to a git repo, and that git repo has both pom.xml and deps.edn files?#2019-08-0802:34andy.fingerhutIf that is involved, no worries, I can wait.#2019-08-0802:34andy.fingerhut(or experiment)#2019-08-0802:37alexmillerdeps.edn is first#2019-08-0802:38alexmillerthat's why I said it matters to tools.dep#2019-08-0802:39andy.fingerhutthx. Keeping some notes separate from here for future reference.#2019-08-0803:03seancorfield@andy.fingerhut When building JARs, you're mostly just going to get src and the META-INF stuff (which includes a pom.xml file) but nothing else from the repo (so no deps.edn file).#2019-08-0803:04seancorfieldWhen you depend on a JAR, the transitive dependencies are figured out from that embedded pom.xml file (as I found out recently with depstar!).#2019-08-0803:05seancorfieldWhen you depend on a git or local repo, tools.deps will use deps.edn if present else pom.xml. If neither are present, you can tell tools.deps to "assume" deps.edn (and it treats the absence as an empty {}).#2019-08-0803:06seancorfield:deps/manifest :deps#2019-08-0803:06seancorfield(see https://clojure.org/reference/deps_and_cli#_dependencies )#2019-08-0803:13andy.fingerhutthx!#2019-08-0815:05hkuptyHey, when using tools.deps on a git dependency (which is a tools.deps project itself as well), can I load an alias of this dependency?
The use-case is: I want to dynamically load a project using tools.deps+dynpath (up to this point I got it running), access its test/ folder and run the tests.
No specific project yet, just trying some ideas.#2019-08-0815:11alexmillerno, not supported#2019-08-0815:11alexmillerbut something under consideration#2019-08-0815:13hkuptyI think that could be solved by adding paths and deps with resolve-deps and make-classpath then.#2019-08-0815:16hkuptyIf I got it right, then calling (make-classpath {...} ["test"] nil) could allow that, right?#2019-08-0815:20alexmillerneed more words to understand what you mean#2019-08-0815:21alexmillerwhat is "that" and "it"?#2019-08-0816:01hkuptyMy intent of being able to run the tests from a dependency I pull dynamically as explained in the use case above.#2019-08-0816:09alexmillerthen, yes that should work#2019-08-0816:09alexmilleryou probably want to also include the source paths though I would assume#2019-08-0816:09alexmillerlike ["src" "test"]#2019-08-0816:12hkuptyGood. Thanks for the help.#2019-08-0818:06jrwdunhamCan someone here help me to understand how I can rebuild/re-install a local dependency without quitting and restarting Spacemacs/Cider? I was originally declaring a Git dependency and then I changed my deps.edn file so that the dependency was to a local path. That is, my deps.edn started out as {:deps github-myorg/mylib {:git/url "" :sha "6c9cbfa1b1721b3d7789e6ff65a3c01065fd6c54"}} and changed to {:deps mylib {:local/root "/path/to/root/of/mylib"}}.#2019-08-0818:58alexmilleryou'll have to restart your repl to pick that up#2019-08-0819:06jrwdunhamthanks @alexmiller. I was thinking that would be necessary, but was hoping not#2019-08-0823:12kennyI have a project that depends on a library A via :local/root. Library A depends on com.google.cloud/google-cloud-monitoring. google-cloud-monitoring depends on commons-logging. In library A, I have :exclusions [commons-logging] set on com.google.cloud/google-cloud-monitoring. When I run clj -Stree in my top level project, I still get commons-logging/commons-logging in the tree. If I run clj -Stree in library A, I also get commons-logging. Any idea what is going on here?#2019-08-0823:12kennyGlobal exclusions would solve this btw ^#2019-08-0823:14seancorfield@kenny commons-logging is used by a lot of things -- perhaps something else is bringing it in?#2019-08-0823:14kennyPerhaps. But it is shown under com.google.cloud/google-cloud-monitoring#2019-08-0823:14alexmillerwhere do you see it in the tree?#2019-08-0823:15alexmillerdid you try :exclusions [commons-logging/commons-logging] ?#2019-08-0823:15kennyWell, several layers deep in there#2019-08-0823:15kennyNo, I can though. I thought they'd be the same.#2019-08-0823:15alexmillerI wonder if there's a non-canonicalized matching canonicalized somewhere#2019-08-0823:15kennycom.google.cloud/google-cloud-monitoring 1.78.0
javax.annotation/javax.annotation-api 1.3.2
com.google.cloud/google-cloud-core-grpc 1.78.0
com.google.auth/google-auth-library-credentials 0.16.1
io.grpc/grpc-protobuf 1.21.0
io.grpc/grpc-protobuf-lite 1.21.0
com.google.api/gax-grpc 1.45.0
io.grpc/grpc-alts 1.21.0
org.apache.commons/commons-lang3 3.5
io.grpc/grpc-grpclb 1.21.0
com.google.protobuf/protobuf-java 3.7.1
io.grpc/grpc-stub 1.21.0
com.google.api.grpc/proto-google-cloud-monitoring-v3 1.60.0
com.google.cloud/google-cloud-core 1.78.0
com.google.api.grpc/proto-google-iam-v1 0.12.0
com.google.protobuf/protobuf-java-util 3.7.1
com.google.api/api-common 1.8.1
com.google.http-client/google-http-client 1.30.1
com.google.j2objc/j2objc-annotations 1.3
io.opencensus/opencensus-contrib-http-util 0.21.0
org.apache.httpcomponents/httpclient 4.5.8
org.apache.httpcomponents/httpcore 4.4.11
commons-logging/commons-logging 1.2
#2019-08-0823:16alexmillerif you can, I'd try changing the exclusion#2019-08-0823:16kennyCommenting google-cloud-monitoring shows another dep bringing it in. I guess I can follow the trail.#2019-08-0823:17kennycommons-logging/commons-logging does it#2019-08-0823:18kennyGosh, I wonder how many other places have an exclusion that isn't qualified and it should be 🤔#2019-08-0823:19kennyWait, now commons-logging is showing up in another location in the tree.#2019-08-0823:20alexmilleryeah, the code doesn't canonicalize the exclusions#2019-08-0823:21kennyChanging the :exclusions in a :local/root dep does not invalidate the clj cache.#2019-08-0823:38alexmillerYeah, that’s a known issue#2019-08-0823:39alexmiller-Sforce#2019-08-0823:42alexmillerI have a plan to fix it but it’s a pain#2019-08-0823:43kennyGotcha, ok. Out of curiosity, would it be possible to resolve deps in parallel?#2019-08-0823:51alexmillerYes, have thought about it some#2019-08-0823:53alexmillerGenerally you have most stuff in local repo already so I don’t find it critical#2019-08-0900:20kennyThe piece that is more interesting to me is shortening the time it takes to get a classpath. We have some services where it takes nearly a minute to get a classpath.#2019-08-0902:22alexmillerwhy?#2019-08-0902:23alexmillerare you i/o bound (and if so, why?) or is the tree huge? or it something in resolution (cpu bound)?#2019-08-0902:44alexmillera minute seems super long so would like to learn more about why#2019-08-0902:45alexmilleralso, curious what version of clj you're on? Just reading the code, it seems like the exclusions should get canonicalized in this scenario, at least in current code#2019-08-0902:47alexmilleroh, I see why#2019-08-0902:56andy.fingerhutAre there transitive dependency reachability checks done using exponential time algorithms? 🙂#2019-08-0902:57andy.fingerhutProbably a completely irrelevant remark -- I just recall finding and fixing such a thing in tools.namespace years back.#2019-08-0903:03alexmillerit's designed to be iterative and should be a single pass over the tree#2019-08-0903:05alexmillerbut problems like missing a cycle could cause issues#2019-08-0903:07alexmiller@kenny I found and fixed the bug with exclusion canonicalization, really a problem with exclusions in any deps.edn, not just transitive. very good catch!#2019-08-0903:07alexmillernot going to release it right now but will be in whatever the next release is#2019-08-0903:25seancorfieldI looked at our deps and we certainly have a mix of bare and canonicalized exclusions so it'll be interesting to see what affect that has on us 🙂#2019-08-0903:47alexmillermy read of current release is that all bare exclusions are being ignored#2019-08-0903:48seancorfieldThanks. I'll try to qualify all of ours and see if anything falls out...#2019-08-0903:51alexmillermight want to do an -Stree before/after#2019-08-0914:02kenny@alexmiller Thanks for the fix!
> are you i/o bound (and if so, why?) or is the tree huge? or it something in resolution (cpu bound)?
Just on a regular computer w/ ssd so not i/o bound. The tree is pretty big but I'd expect a tree this size with any enterprise product. Could be something in resolution that is slow, don't think it's cpu.#2019-08-0914:05alexmillerI mean i/o bound in talking to the network to download jars#2019-08-0914:05alexmillernot filesystem#2019-08-0914:05kennyOh, no. I think everything is already on disk#2019-08-0914:06alexmillerlike during that minute, could you grab periodic stack traces, either with ctrl-\ or with jstack, and look at the top of the stack?#2019-08-0914:06kennySure#2019-08-0914:07alexmillerso parallel downloads would not help you at all if you're not downloading anything#2019-08-0914:07kennyRight#2019-08-0914:08alexmilleranother debug thing to do is -Sdeps '{:aliases {:v {:verbose true}}}' -A:v#2019-08-0914:08kennyBTW, the slowness is a "full" refresh, not using the cache.#2019-08-0914:08kennye.g. adding/changing/removing a dep#2019-08-0914:08alexmillerif you see pauses in there, that would be suspicious, but otherwise, if you just have a big trace, I'd be interested in seeing that, could dm it to me#2019-08-0914:09alexmillernot using which cache? classpath cache? m2 local repo? gitlibs cache?#2019-08-0914:10kennyNot sure. Changing certain deps.edn takes a long time.#2019-08-0914:10alexmillerso you're not actively clearing the m2 repo or anything#2019-08-0914:10kennyOh, no.#2019-08-0914:10kennySame for gitlibs and the rest#2019-08-0914:10alexmillerso really classpath cache#2019-08-0914:11alexmillerthat's the only thing stale when you change deps.edn#2019-08-0914:12alexmillerand you're on latest clj?#2019-08-0914:12alexmillerthere were some cycle detection issues that were fixed months ago#2019-08-0914:12alexmillerclj -Sverbose for version#2019-08-0914:12kennyThis is from -Sdescribe :version "1.10.1.466"#2019-08-0914:13alexmilleryeah, that's latest#2019-08-0914:13alexmillerwell, I'd love to take a look#2019-08-0914:16kennyFor starters, these messages have reappeared and they take a decent chunk of time:
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-netty-shaded/maven-metadata.xml from
@seancorfield had noticed that a pom from one of my deps (com.google.cloud/google-cloud-monitoring "1.78.0") used a RELEASE version. He suggested explicitly specifying the deps mentioned there. I have done that and they still appear.#2019-08-0914:17alexmilleryeah, that's actually an s3 wagon issue, upstream from tools.deps#2019-08-0914:18kennyHe also mentioned it may have something to do with not correctly resolving deps from a parent pom.#2019-08-0914:18kennyOh. Well that could easily shave ~15s off the time.#2019-08-0914:18kennyThose deps are coming from google-cloud-monitoring not Datomic#2019-08-0914:26alexmillerI don't have enough info to debug this, would be useful to see deps.edn and the verbose trace above#2019-08-0914:28kennySure. I can send that over. I'll see if I can create a smaller deps.edn first.#2019-08-0914:29kennyJust this will do it:
{:deps {com.google.cloud/google-cloud-monitoring {:mvn/version "1.78.0"}}
:mvn/repos {"datomic-cloud" {:url ""}}}
#2019-08-0914:31alexmillerthis is starting to ring a bell#2019-08-0914:32alexmillerthere's a loop in these maven deps iirc#2019-08-0914:37kennyAdding time to the clj calls show I drastically overestimated the time it takes:
real 0m23.962s
user 0m46.126s
sys 0m1.158s
This certainly feels like an eternity when needing to do that many times a day. A big portion of that time is the "Downloading: ..." thing. It would be a huge productivity boost to get that under 5s.#2019-08-0914:38ghadiwhy does repeated downloading happen in your environment?#2019-08-0914:38alexmillerthat's an issue with the s3 wagon I think#2019-08-0914:38ghadi(jumping into this conversation without reading the backscroll)#2019-08-0914:38kennyDoes it not do that for you?#2019-08-0914:38alexmillerI can repro it#2019-08-0914:39alexmillerwithout the datomic repo in the mix, it's about 5 seconds to build a classpath for that#2019-08-0914:40kennyThat s3-wagon thing has always been a nightmare for me. I remember always hitting issues with it back when we used an s3 maven repo. Perhaps a good use case for aws-api? 🙂#2019-08-0914:40alexmillerwith it, I see about 8-9 seconds#2019-08-0914:41kennyYeah, I'd be curious what one of our large apps takes without the downloading thing.#2019-08-0914:41alexmillertime clj -Spath -Sforce#2019-08-0914:41kennyThat will still have the downloading issue.#2019-08-0914:42alexmilleryeah, that's the idea :)#2019-08-0914:42kennyOh. I already sent that above haha#2019-08-0918:31ikitommiany success stories of multi-module/mono-repo library setups with deps? have some working lein projects for that, but have now a deps project that needs to be split into parts.#2019-08-0918:46kenny@ikitommi We switched to a monorepo a month or so ago. All internal libraries are :local/root. Makes working in the REPL great. There's a few kinks with our setup though:
- We use CircleCI for CI/CD. There's no support for monorepos with CircleCI. This leads to longer build times - every project runs through its test steps with every push. We recently switched to their new unlimited parallelism plan which has been quite helpful in getting CI time down.
- CI configuration was moved into a set of clojure files because it became far too tedious messing around with YAML with the number of projects we have. This does, unfortunately, mean that CI config needs to be manually generated with a command every time you change the CI clojure files.
- We have a small service diff library that detects when a particular service's code, deps.edn, or :local/root deps have changed. That ensures a service won't get deployed on every push.
- We don't have a great way to have a common deps.edn across all projects. This would be quite useful for things like: global exclusions (when supported), overriding certain library versions, common aliases, etc. Ideally there'd be some way to just pass in N number of deps.edn files to clj and have it merge those in. I use Cursive so Cursive would also need to have some way to select which deps.edn files to use.
- We don't have a good way to run commands across all projects or only in a certain project. For example, I'd like to be able to do something like: monorepo my-service uberjar.
- Different libraries & services run tests with a different set of aliases. Every time we want to run the tests for a project, you need to go to the projects README (or check the CI config) and determine which aliases to use to run the tests. Either the aforementioned "command runner" or a way to combine aliases somehow would make this much better.
Overall, this workflow is far better than having individual repos and constantly needing to restart the REPL when working across projects.#2019-08-0919:04ikitommiThanks @kenny! was hoping for the monorepo kinda script, too lazy to start cooking up own tools right now. In my case, it's a library, going to be split into set of libraries, so would be easy to have same aliases for all. A sample repo would be super awesome.#2019-08-0919:11dominicm@ikitommi you might want to look at edge which does this#2019-08-0919:12dominicmAlso Sean has talked a lot about their setup at world singles#2019-08-0919:17seancorfieldWe have a monorepo with maybe two dozen subprojects, and 90k lines of Clojure.#2019-08-0919:18seancorfieldThe key thing we did was to have a primary deps.edn in a folder and point to that via CLJ_CONFIG (so it "replaces" the user-level deps.edn) and then each subproject has a deps.edn.#2019-08-0919:19seancorfieldWe use :override-deps in the primary file to "pin" versions of libs across the whole repo as needed, as well as provide all the common tooling via aliases.#2019-08-0919:19seancorfieldThe only "tooling" we've built on top of this is a small shell script that can execute multiple clojure commands and knows how to navigate to subprojects when running series of commands.#2019-08-0919:21seancorfieldLike @kenny we use :local/root deps for cross-module deps -- and we have an everything subproject that we can build the deps.edn into from across the monorepo and that's where we usually start our REPL/REBL from.#2019-08-0919:22kennyHmm that's a good idea! Cursive doesn't support CLJ_CONFIG unfortunately. Creating a everything project and starting a nrepl from the command line could solve that problem though! Generally it makes sense to have everything on the classpath while dev'ing.#2019-08-0919:23kennyHow does your script know which aliases to use for each project's tests?#2019-08-0919:34seancorfieldbuild set:of:aliases subproject is our shell script. But it can take multiple pairs of aliases/subprojects.#2019-08-0919:34seancorfieldand if we need arguments, we can use [ ] to wrap them, so build uberjar api [ run ci-ftp api #2019-08-0919:43kennyHow do expose build? Is it a script at the root of the repo? Do you have devs add to PATH?#2019-08-0919:55seancorfieldWe have a <repo>/build/bin folder containing scripts. Devs can either add it to their path or just run the scripts directly.#2019-08-0919:56seancorfieldI mostly work in the build folder but I have build/bin/build symlinked into my ~/bin folder for convenience. Other stuff I run with ./bin/<script>#2019-08-0919:57seancorfieldBetween docker compose and two git clone commands, a dev can be set up "immediately" (assuming they have an OpenJDK8 installed).#2019-08-0919:58seancorfieldIf they need to work on our legacy apps, there's one more git clone to run.#2019-08-0919:59seancorfield(one of those repos is for semi-static tooling, which is where we run docker compose -- for Redis, Elastic Search, Percona/MySQL, and a custom search engine we use)#2019-08-0920:11nwjsmithI’ve been using this trick to “bless” <repo>/bin path additions: https://thoughtbot.com/blog/git-safe#2019-08-0920:17seancorfieldInteresting little trick!#2019-08-0922:22kennyDoes CLJ_CONFIG have to be an absolute path?#2019-08-0922:23kennyI think it doesn't. I thought it wasn't working for a sec.#2019-08-0922:23seancorfieldNope. We use "../versions" in our script.#2019-08-0922:23kennyHow do you deal with not knowing where the build script is run from?#2019-08-0922:24seancorfieldCLJ_CONFIG=../versions clojure -A:defaults:<task> <args>
:defaults pulls in all the overrides etc from versions/deps.edn#2019-08-0922:24kennyFor example: if I have a build/bin/build and I run it from build/bin, I need to know to set CLJ_CONFIG to ../../versions. If I run it from build, I need to set CLJ_CONFIG to ../versions.#2019-08-0922:25seancorfielddirname $0#2019-08-0922:26seancorfieldThen you can work relative to that.#2019-08-0922:26kennyOooo, I didn't know about $0#2019-08-0922:27seancorfield(although we assume certain filesystem paths are the same on all dev/test/prod images so some of our scripts take advantage of that -- and devs just add a symlink to wherever they decided to put stuff)#2019-08-0922:50kennyGiven you symlink build, dirname $0 will return a path to wherever the symlink is. How do you deal with that?#2019-08-0922:51kennyPerhaps readlink?#2019-08-0923:20seancorfieldYes, you can use readlink $0 to get the actual file location (it exits with a non-zero status if the argument is not a symlink).#2019-08-0923:21kennyYou must then also deal with Linux/Mac platform differences with readlink 😵#2019-08-0923:22seancorfieldLike I said above, we also assume certainly filesystem paths to make our lives easier 🙂#2019-08-0923:23seancorfield(partly because we have a shell script in the main repo that a new dev can download and it does most of the env setup for them, including git cloneing repos to specific places and setting up symlinks and handling the initial Mac/Linux differences)#2019-08-0923:24seancorfieldBut there are plenty of ways to skin that particular kitty.#2019-08-0922:59hkuptyWhat is the difference between:
• (resolve-deps {:paths [p1 p2] ...}),
• (make-classpath ... [p1 p2]) and
• (make-classpath .. nil {:extra-paths [p1 p2]})
All three seem to produce the same result, which is to add the local project's paths p1 and p2 to the classpath.
To further exemplify my question from yesterday, I'm looking for something on the lines of:
{:deps {some/dep {:git/url ... :paths [p1 p2]}}}
Is the example above feasible?#2019-08-0923:31alexmillerThose 3 produce the same result but are semantically different#2019-08-0923:32alexmillerWhat’s your goal?#2019-08-0923:33hkuptyI want to add a folder (for example 'test') from a dependency I loaded w/ tools.deps to my currently running project's classpath.#2019-08-0923:34hkuptyThat folder is not declared on the main {:paths [...]} clause on dependency's deps.edn for obvious reasons, it is not the mainline for that project.#2019-08-0923:36hkuptyHowever, as I'm trying to build something like a buildscript CLI interface for a group of projects, I want to dynamically load them using tools.deps and run their tests, or whatever else I might want to do with them, given that not only I can load their dependencies but also add folders (or other aliases from that deps.edn) into this buildscript CLI classpath.#2019-08-0923:46hkuptyThere's a hacky way for me to work around that, which is to hijack the result of (result-deps) through some (update-in (result-deps ...) [dep :paths] conj '/my/hand-crafted/path/p2') before (make-classpath ...). That seems too hacky for me, but I can do that if tools.deps doesn't want to explore further the project structure of a dependency.#2019-08-0923:48alexmillerIf you’re calling tools.deps programmatically then you’re already in the machine - do whatever you want with the intermediate results#2019-08-0923:52kenny@ikitommi I added a build script to our repo that is similar to the one @seancorfield described. Here's what I ended up with. Our repo is structured like with all projects under projects and this script located at bin/build.#2019-08-1000:03seancorfieldI should probably see if I can synthesize a generic version of our build script and release it as open source...#2019-08-1000:04kennyThat would be super awesome 🙂 I just realized mine doesn't support aliases very nicely...#2019-08-1000:09seancorfieldWe have a whole bunch of WS-specific alias expansions in it right now. build run thing will expand into whatever thing means in our project. So build run database-setup expands to build [ migrate migration photos dev ] which runs the :migrate alias in the migration subproject and then runs the :photos alias in the dev subproject.#2019-08-1000:12seancorfieldWe also have an expand-deps.sh script that looks for :local/root deps in a subproject's deps.edn file and recursively expands the list of subprojects to include all transitive subprojects that are depended on -- so we can say build test-all api and it will expand to build tests environment datamapper worldsingles ... api and that in turn expands to running build test <subproject> for each subproject in turn 🙂#2019-08-1000:13seancorfieldprev=""
curr="
where $clojure is the monorepo root.#2019-08-1000:15seancorfieldAll told, including the WS-specific task alias expansion and the 70 lines of help output, we're talking 350 lines of bash.#2019-08-1000:19kennyHmm, that's a lot of bash haha. I just about hit my tolerance with the wee 16 lines above 😛#2019-08-1000:24seancorfieldI've created some truly monstrous bash-based systems in my career... 1,000's of lines long. I probably wrote my first *nix shell script back in '79 tho'... neckbeard#2019-08-1000:27kennyWriting a small script to do the necessary parsing in plain ol' Clojure only adds 1s overhead 🙂#2019-08-1000:37seancorfieldTrue, I could rewrite all this in Clojure (and shell out to clojure as needed for the repeated invocations). I could make all the WS-specific stuff data driven and external too. Maybe one day 🤔#2019-08-1000:59alexmiller@kenny I made some progress on your perf issue today#2019-08-1000:59alexmillerThere’s some repository cache stuff built into Maven that we weren’t using so I’ve changed that and it makes a big difference.#2019-08-1001:01alexmillerThere’s still something I don’t understand about why you’re getting that s3 maven download every time, but it’s now only doing it once at least. #2019-08-1001:08kennyAwesome! Is there a way for me to test it out before a release?#2019-08-1001:13seancorfield@kenny Looking at the clojure script, I think the answer is "not easily".#2019-08-1001:16seancorfieldYou'd have to build tools.deps JAR yourself and overwrite the version in the clojure install or hack the clojure script to point to that JAR elsewhere#2019-08-1001:21kennyThat doesn’t sound that bad. Perhaps writing bash makes tasks like that sound easy haha#2019-08-1001:32alexmillerYou don’t even need to build it#2019-08-1001:33alexmillerJust clone the repo#2019-08-1001:35alexmillerThen modify your clojure script to put the tools.deps.alpha/src/main/clojure dir at the front of the tools_cp var#2019-08-1001:38alexmillerOr you can just wait a day or two for me to test and release it :)#2019-08-1006:05dominicmThere's one downside to not building it since the removal of the system deps.edn, you end up with templated version strings! So you need a hard clojure dependency.#2019-08-1012:39alexmillerAh, true. Well you can build and use the target output, that should work.#2019-08-1015:54alexmillerSome tools-deps updates here: http://insideclojure.org/2019/08/10/journal/#2019-08-1204:38onetomis there a definitive guide on how to combine some java sources into your deps.edn clojure project?
like which directories should i put my java sources?
when and how to run the compilation into .class files?
where should i put the resulting class files, so i can import them from clojure?
things like that.#2019-08-1204:39onetomi have some protobuf .java files from a decompiled android app and i would like to use it via clojure repl...#2019-08-1204:42onetomi found this old lein docs:
https://github.com/technomancy/leiningen/blob/master/doc/MIXED_PROJECTS.md
so i can rephrase my question as:
what's the equivalent of
1. :java-source-paths?
2. lein javac?
i don't need clojure :aot, because im not accessing clojure code from java,
plus that feature has a lot of docs available already..#2019-08-1204:47onetomi do need to access the same maven deps from java which i specified in deps.edn though...
i know from the command line i can just do java -cp $(clj -Spaths)...,
but do i need something like this for javac compilation too?
( plus i what i really don't have an idea about is how to make symbols resolve in cursive in java sources; but that i can ask on #cursive )#2019-08-1204:48cfleming@onetom For source paths, you can just use :extra-paths. Once you have that set up, Cursive will just work.#2019-08-1204:49onetomcan i just mix the java source files into the same src/ directory as the clojure files then?#2019-08-1204:49cflemingThere’s no lein javac equivalent in deps. You’ll need to set up aliases for compilation and runtime - compilation will have your Java sources on the classpath, and runtime will have the resulting class files.#2019-08-1204:49cflemingYou can mix them, yes, but your Java source files will be on the runtime classpath then.#2019-08-1204:50cflemingI’d keep them separate if I were you, it’s trivial to do so.#2019-08-1204:50cflemingYou’ll also need a run alias to do the actual compilation, but I’ve not investigated that.#2019-08-1204:51onetomah, i see.
so is there any templates out there which showcase some "recommended/reasonable" setup?#2019-08-1204:51cflemingThere’s no concept of automatically compiling before run with deps, so you’ll need to manually recompile if anything has changed.#2019-08-1204:52cflemingYou could set that up in IntelliJ and just let IntelliJ compile for you, depending on how official or serious your project is#2019-08-1204:52onetomthe built-in Build feature would be intuitive to be used for this purpose#2019-08-1204:52cflemingYes, you can do that. You could just add a Build Project before-run step to your run configuration.#2019-08-1204:53cfleming#2019-08-1204:54cflemingSo to set that up set up Java compilation as normal, and add an alias to your project which adds the compile output path to your classpath.#2019-08-1204:54onetomthanks a lot! i will try it.
i hope i don't have to pull in some build tool for this simple java compilation 🙂#2019-08-1204:55cflemingAdd another alias with your extra source paths, then select it for project sync, that will mean that Cursive will add that path as a source path for working with the sources: https://cursive-ide.com/userguide/deps.html#working-with-aliases#2019-08-1204:56cflemingThen in your run config, run with deps and use your runtime alias to add the compilation output to the classpath.#2019-08-1204:56cflemingAnd you should be good to go 🙂#2019-08-1204:57onetomi only understood half of it, but haven't seen your deps.html documentation yet, so it will be clear afterwards, im sure#2019-08-1204:57cflemingOk, ask over in #cursive if you have problems, it’s more of a Cursive thing than a deps thing.#2019-08-1204:58onetombtw, i've just upgraded to IntelliJ 2019.2.1 Preview last night and indeed Cursive deps support works without issues with it so far!#2019-08-1204:59cflemingYes, they fixed a few problems there, but there are still more. I’m actually in the process of migrating the deps support off the IntelliJ API I was using, and instead basing it on the lein support, so I’ll be insulated from all their platform issues.#2019-08-1211:01kvltI've noticed since moving to deps.edn my log lines no longer show up in my emacs repl buffer. Anybody know why this is?#2019-08-1211:05kszabohttp://insideclojure.org/2019/08/10/journal/#toolsdeps#2019-08-1211:05kszabo> And there have been some complaints that tools.deps includes slf4j-nop as a dep. slf4j is a pluggable logging framework and the nop logger is a “don’t log” plugin, which takes away the choice for library consumers to plug as they wish. tools.deps will remove that dependency but it will still be included in the clj installation.#2019-08-1211:14kvltThank you! So I should just be able to add the dependency and it should log again?#2019-08-1213:21alexmillerjust using clojure to build a classpath should not affect the clojure program you run with the classpath after that (as tools.deps is not included in that).#2019-08-1213:21alexmillerand that's how I'm interpreting your problem statement at the top#2019-08-1213:22alexmillerit's possible that cider is also relying on tools.deps for something and that is affecting your program somehow, but I don't know enough about how or whether cider uses tools.deps to know if that could be related#2019-08-1213:24alexmillerI would start by opening an issue or asking a question in #cider before jumping to the conclusion that this is related#2019-08-1411:02unbalancedSo I'm looking for a command line solution to handling generic tooling tasks such as "bump version number", or "modify deps.edn to add something to :deps" ... do we just write our own scripts to do that sort of thing or is there a canonical solution?#2019-08-1411:05dominicmhttps://github.com/clojure/tools.deps.alpha/wiki/Tools#versioning-and-releasing should help you 🙂#2019-08-1411:06dominicm(and lots of other useful tools there too)#2019-08-1411:06unbalancedAlso I'm still really curious what the heck a build.clj file is, such this:
(require '[cljs.build.api :as b])
(b/build "src"
{:output-dir "out"
:output-to "out/main.js"
:optimizations :none
:main 'example.core
:install-deps true
:npm-deps {:react "15.6.1"
:react-dom "15.6.1"}})
reference here java -cp ls ../../lib/*.jar | paste -sd ":" -`:../../src/main/cljs:../../src/main/clojure:src clojure.main build.clj`
with tools-deps you can do this though: clojure build.clj#2019-08-1411:11unbalanced2017 was rough 😬#2019-08-1411:12dominicmprobably why so many used lein#2019-08-1411:19unbalancedIs there a place in deps.edn where we put license/version info or does that go somewhere else now?#2019-08-1411:20dominicmsomewhere else. pom.xml like before would be fine I think.#2019-08-1411:22unbalanced👍#2019-08-1415:31dominicm@alexmiller in tools.deps.alpha.reader, is it worth having the user-deps locator as a separate function? In pack the issue has been raised to include part of the user deps.edn which seems acceptable. That part is just for including maven repo settings which are user-local.#2019-08-1415:33alexmillerSure#2019-08-1415:35dominicmwill open a jira#2019-08-1506:38murtaza52is there any tooling for auto completion built on top of the tools.deps repl (started using the clj command) ?#2019-08-1506:47seancorfieldThe Rebel Read line tool might be what you're looking for. There is an alias for it in my dot-clojure repo deps.edn#2019-08-1510:11murtaza52thanks going through it - https://github.com/seancorfield/dot-clojure#2019-08-1616:18zaneThere is also lots of editor-specific tooling for auto-completion that plays well with tools.deps-based projects.#2019-08-1808:47murtaza52is there a lein new x equivalent for tools.deps#2019-08-1808:57dominicmClj-new#2019-08-1808:58andy.fingerhutSee here for a list of that and other tools.deps related tools: https://github.com/clojure/tools.deps.alpha/wiki/Tools#2019-08-1810:26murtaza52thanks guys#2019-08-1813:08murtaza52going through the docs, not able to understand the significance of the -A alias, when should it be used and when not ? Bcoz it seems it can do what all other aliases can do and vice versa, so either only it is needed or others are needed.#2019-08-1813:26alexmillerGenerally you can just use -A#2019-08-1813:27alexmillerThere are some weird cases where you might only parts of an alias to take effect#2019-08-1814:37murtaza52thanks, that explains it#2019-08-2010:33unbalancedis there any way that you can specify extra deps on the command line without using an alias?
I've developed a template to help my coworkers quick-start on clojurescript but explaining to them how to add an alias for clj-new to their .clojure/deps.edn is just one more annoyance for them to figure out. I'd like to be able to do something like
clj -m clj-new.create --middleware "[seancorfield/clj-new]" some.example
#2019-08-2017:43souenzzowhy not clj -Sdeps '{:deps,{seancorfield/clj-new,{:mvn/version,"0.5.5"}}}' -m clj-new.create some.example#2019-08-2022:01unbalancedthat's it!#2019-08-2010:34unbalancedor perhaps read a deps.edn from stdin that I could have them just paste in via file descriptor?#2019-08-2010:37unbalancedclj -i -m clj-new.create some.example <(cat << eof
{:aliases :new {:extra-deps {seancorfield/clj-new
{:mvn/version "0.5.5"}}
:main-opts ["-m" "clj-new.create"]}
eof
)
something like that would also work but I haven't been able to figure that one out yet either#2019-08-2010:43vlaaadit's worth explaining IMHO: there is no magic, everything is simple and configurable#2019-08-2011:28alexmiller-Sdeps lets you add an extra deps.edn on the command line#2019-08-2019:05unbalanceddoes anyone have an example of the -Sdeps syntax? Is a string?#2019-08-2019:07lispycloudsclj -Sdeps "{:deps {clj-http {:mvn/version \"3.10.0\"}}}" ?#2019-08-2019:07unbalancedthe command I'm hoping for is
clj -Sdeps '{seancorfield/clj-new {:mvn/version {"0.5.5"}}' -m clj-new.create project.name#2019-08-2019:07unbalancedahhh ok#2019-08-2019:07unbalancedthanks I wasn't sure the exact syntax#2019-08-2019:17alexmillerit is literally the text of a deps.edn file#2019-08-2021:44unbalancedoutstanding ... exactly what I needed, works exactly as advertised, thank you!#2019-08-2022:32cflemingI’m having a problem loading a namespace which requires t.d.a, using version 0.7.535. I’m getting the following error:
Syntax error compiling at (clojure/tools/deps/alpha/reader.clj:98:5).
No such var: io/read-edn
#2019-08-2022:32cflemingThis seems weird, because io/read-edn does exist.#2019-08-2022:33cflemingI’m just running a standard REPL, classpath looks ok to me.#2019-08-2022:35cfleming( "clojure/tools/deps/alpha/reader.clj")
=>
#object[java.net.URL
0x2bf95d38
"jar:file:/Users/colin/.gradle/caches/modules-2/files-2.1/org.clojure/tools.deps.alpha/0.7.535/11de58e45ac07ea473f2322f335ef6d162dffde9/tools.deps.alpha-0.7.535.jar!/clojure/tools/deps/alpha/reader.clj"]
( "clojure/tools/deps/alpha/util/io.clj")
=>
#object[java.net.URL
0x2b2beaaa
"jar:file:/Users/colin/.gradle/caches/modules-2/files-2.1/org.clojure/tools.deps.alpha/0.7.535/11de58e45ac07ea473f2322f335ef6d162dffde9/tools.deps.alpha-0.7.535.jar!/clojure/tools/deps/alpha/util/io.clj"]
#2019-08-2022:39hiredmanmaybe check (get (ns-publics ') 'read-edn)#2019-08-2022:39hiredmanthe namespace could have been created somehow and added to the lib-map, without loading the source file, stopping require from actually loading it#2019-08-2022:39cfleming(get (ns-publics ') 'read-edn)
=> nil
(ns-publics ')
=>
{printerrln #',
write-file #',
slurp-edn #'
#2019-08-2022:40cflemingAll the vars are there except that read-edn#2019-08-2022:42hiredmanno install-deps either (not sure if that is in 0.7.535, I am looking at master)#2019-08-2022:42cflemingNo, I don’t see that in the 0.7.535 rev#2019-08-2022:44cflemingI can’t see anything in the source doing anything funky like unmapping vars.#2019-08-2022:44hiredmanread the diff wrong, install-deps is a different namespace#2019-08-2022:45cflemingIf I just restart the REPL and then require http://c.t.d.a.u.io directly, then the publics are wrong right off the bat.#2019-08-2022:46cflemingIs there a good way to diagnose where the ns is actually being loaded from?#2019-08-2022:48hiredmanyou could change the require function to print debugging information(a stacktrace, maybe clojure.lang.Compiler/LOADER, but I guess that is just a dynamic classloader) every time it is called#2019-08-2022:51hiredmanyou could also see what (.getResource (RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj") returns, maybe check for aot'ed classfiles too#2019-08-2022:51cfleming(.getResource (RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj")
=>
#object[java.net.URL
0x37339a
"jar:file:/Users/colin/.gradle/caches/modules-2/files-2.1/org.clojure/tools.deps.alpha/0.7.535/11de58e45ac07ea473f2322f335ef6d162dffde9/tools.deps.alpha-0.7.535.jar!/clojure/tools/deps/alpha/util/io.clj"]
#2019-08-2022:51cflemingThe same as before, looks like it’s coming from the right jar.#2019-08-2022:52hiredmanyou could also look in the jar and see if has the right file contents 🙂#2019-08-2022:53cflemingIt does 🙂#2019-08-2022:58hiredmansomething like (alter-var-root #'require (fn [orig]
(fn [& args]
(println (.getResource (clojure.lang.RT/baseLoader) "clojure/core.clj"))
(apply orig args))))
with the funky namespace's resource name instead of clojure/core.clj and then requiring the namespace might show something#2019-08-2023:00cfleming(alter-var-root #'require (fn [orig]
(fn [& args]
(println (.getResource (clojure.lang.RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj"))
(apply orig args))))
=> #object[user$eval1256$fn__1257$fn__1258 0x1a272414 "#2019-08-2023:01cflemingI’m not sure why require would be called twice there.#2019-08-2023:03hiredmanthat may be a clue#2019-08-2023:03cflemingAnd if I use:
(alter-var-root #'require (fn [orig]
(fn [& args]
(println (slurp (.getResource (clojure.lang.RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj")))
(apply orig args))))
then the correct file contents are printed out.#2019-08-2023:03hiredmanI think I've seen that behavior when the namespace in question has already been loaded (why that results in two require calls I have no idea)#2019-08-2023:03cfleming(twice)#2019-08-2023:04hiredmanin a fresh repl before requiring anything maybe try the ns-publics call for the namespace again#2019-08-2023:05cflemingIn a plain clj -Adeps REPL, where the publics are correct, it’s also printed twice.#2019-08-2023:05cflemingYeah, (find-ns ') returns nil in a fresh REPL.#2019-08-2023:09cflemingWeirdly though, requiring something like clojure.zip only prints once.#2019-08-2023:09alexmiller-Adeps loads an uberjar#2019-08-2023:09alexmillerThat won’t play nice with the lib jar#2019-08-2023:09cflemingNo, I was just using -Adeps without a deps.edn to test in a clean REPL.#2019-08-2023:10cflemingi.e. to compare to what I was seeing in the problematic one.#2019-08-2023:11alexmillerNothing unusual about how this ns is defined or loaded afaik #2019-08-2023:11cflemingNo, I can’t see anything.#2019-08-2023:13hiredmancould you have some aot classfiles somewhere on the classpath, if they have a newer last modified they will be loaded instead of the clojure source#2019-08-2023:14cflemingI’m wondering that. I have the debugger set up, and Compiler.load() is never called.#2019-08-2023:14alexmillerI don’t think that fn was added till 541#2019-08-2023:16cflemingUgh, that was the problem.#2019-08-2023:17cflemingRogue AOT classes sitting around.#2019-08-2023:17cflemingSorry for the noise.#2019-08-2023:17cflemingThanks for the help!#2019-08-2023:18cflemingI’m not sure why there would have been AOT classes for the .io namespace but not the one calling it, though. I guess it must have been a previous transitive compilation.#2019-08-2120:44alexmillerclj 1.10.1.469 and tools.deps.alpha 0.7.541 released...#2019-08-2120:44alexmiller- FIX: exclusions were not canonicalized and could fail to match and exclude
- PERF: cache Maven session and use Maven session cache
- FIX: Remove slf4j-nop as dependency of tools.deps.alpha
#2019-08-2120:47alexmillerif you're a user of tools.deps.alpha, you can now define Maven session boundaries around your use of tools.deps.alpha by wrapping clojure.tools.deps.alpha.util.session/with-session around one or more calls into tda (see https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/script/make_classpath.clj#L58 for example)#2019-08-2120:48alexmilleryou may also need to add slf4j-nop as a dep if you were previously relying on getting it via tools.deps.alpha#2019-08-2123:36seancorfield@alexmiller Under what circumstances might I expect with-session to make a difference?#2019-08-2123:37seancorfieldI just updated clj-new to use the latest t.d.a and wrapped the pair of t.d.a calls in that, but saw pretty much no difference in time spent generating a template (from an empty M2 repository each time). (session/with-session
(-> (deps/resolve-deps deps resolve-args)
(deps/make-classpath (:paths deps) {})
(str/split (re-pattern java.io.File/pathSeparator))
(->> (run! pom/add-classpath)))))
#2019-08-2123:49alexmillerIt will only help if you’re talking to repos of course#2019-08-2123:50alexmillerBut in particular it makes a difference taking to s3 as it’s slow#2019-08-2123:51alexmillerFor building a large classpath (without cp cache), when talking to s3 it was as much as 50% better#2019-08-2123:51alexmillerAnd in other cases, no observable difference#2019-08-2123:54seancorfieldAh, OK. So it's harmless at worst, and might be helpful for some small class of clj-new users...?#2019-08-2203:21alexmilleryes, should be no worse, and occasionally much better#2019-08-2221:42danielcomptonhttps://ask.clojure.org/index.php/8440/equivalent-of-leiningens-managed-dependencies-in-deps-edn#2019-08-2221:46dominicmAfk, but default deps come to mind#2019-08-2221:54danielcomptonThanks! :override-deps also look promising#2019-08-2221:57dominicmThat sounds right too#2019-08-2221:57dominicmI think the downside is the alias requirement#2019-08-2221:59danielcomptonYeah, was just going to say that#2019-08-2222:00danielcomptonDoesn't communicate it in the same way that managed dependencies does#2019-08-2222:02alexmillerdefault-deps is addressing the same use case as dependency management but there is still work to do to make it actually useful#2019-08-2222:29danielcomptonCan you use it at the root of a deps.edn or does it require an alias?#2019-08-2223:05alexmillerAlias #2019-08-2223:42seancorfield@danielcompton I added an answer explaining how we use :override-deps at World Singles Networks -- because we use it extensively!#2019-08-2223:43danielcomptonThanks to both of you! This feels like a great use-case for Ask Clojure to me#2019-08-2223:45seancorfieldhttp://ask.clojure.org is my home page in my browser 🙂#2019-08-2316:40aaelonyI shared this link at work and the feedback was that it is a PHP site...#2019-08-2316:50seancorfieldAlex has addressed that already. It is an off-the-shelf solution that is easy to maintain/administer.#2019-08-2316:50seancorfieldAnd provides all the functionality needed for the task. There is no Clojure/Script equivalent to use (but folks are welcome to write such things and offer them up to Cognitect/the public).#2019-08-2317:07aaelonymakes sense to me#2019-08-2314:53astrasheIs there an example anywhere that shows how to use deps.edn with a local java library? I need a local java library to deal with a protobuf#2019-08-2314:59andy.fingerhutIf the Java library is compiled to a JAR file, then you should be able to add it as a local dependency using something like this: {:deps {somejavalib {:local/root "/path/to/the/jar/file.jar"}}} If you search for ":local" on the reference doc page for deps, you should find that, too: https://clojure.org/reference/deps_and_cli#2019-08-2315:00astrashe@andy.fingerhut Thanks! The JAR file seems like the way to go#2019-08-2315:03vlaaadI wish future clojure releases focused more on java interop, particularly around proxies and class creation. tools-deps is a great value proposition, with the downside being that using .java files in the same project is clumsy. I really don't want to go back to lein...#2019-08-2315:04bronsahttps://clojure.atlassian.net/browse/CLJ-1255#2019-08-2315:21alexmillerI think that's unlikely given that Clojure is in philosophical opposition to concrete derivation#2019-08-2315:23alexmillerother than https://dev.clojure.org/jira/browse/CLJ-1255 which is in consideration for 1.11#2019-08-2315:33vlaaadI think that's what I need actually, having troubles with proxy not having access to some protected stuff and using reflection without type hinting this..#2019-08-2407:44dharriganIs there any chance that "add-libs" branch on tools deps can be rebased from master so that the latest changes on master for the just released version can be brought in? The "add-lib" functionality is really really useful to have (along with the removal of the slf4j-nop library....)#2019-08-2407:44dharriganIs there any chance that "add-libs" branch on tools deps can be rebased from master so that the latest changes on master for the just released version can be brought in? The "add-lib" functionality is really really useful to have (along with the removal of the slf4j-nop library....)#2019-08-2417:00just.sultanovAlso waiting for these changes. I had to select a separate library with small edits for my needs. If interested, https://github.com/just-sultanov/clj-deps#2019-08-2417:03seancorfield@U064X3EF3 is usually pretty good about merging master back to the add-lib branch -- esp. once we add-libbers ask for it 🙂#2019-08-2421:18alexmillerI merged it#2019-08-2421:18alexmillermerged master to it that is#2019-08-2500:09seancorfieldYou rock! Thank you!#2019-08-2500:10seancorfield@U11EL3P9U @U1EQNSHL4 SHA 148cab8f154711f8495bd632ebdc53a14dff7f09 to save you both going to look it up. I'm updating my dot-clojure repo right now...#2019-08-2506:44dharriganw00t w00t!#2019-08-2510:40just.sultanovthank you @U04V70XH6 @U064X3EF3#2019-08-2510:53just.sultanovIn my current project, I need to add libraries to the classpath not only in the REPL at development time. Therefore, I made small changes to the function add-loader-url.#2019-08-2510:56just.sultanov(defn- add-loader-url
"Add URL string or URL to the highest level DynamicClassLoader or URLClassLoader URL set."
[url]
(let [url (if (string? url) (URL. url) url)
thread (Thread/currentThread)
cl (loop [loader (.getContextClassLoader thread)]
(let [parent (.getParent loader)]
(if (instance? DynamicClassLoader parent)
(recur parent)
loader)))]
(if (instance? DynamicClassLoader cl)
(.addURL ^DynamicClassLoader cl url)
(let [loader (.getContextClassLoader thread)
urls (into-array URL [url])
cl (URLClassLoader. urls loader)]
(.setContextClassLoader thread cl)))))#2019-08-2407:44dharrigan🙂#2019-08-2407:45dharriganThank you 🙂#2019-08-2421:04seancorfieldseancorfield/clj-new 0.7.8 -- minor maintenance release: fixes -v / --verbose option (betcha didn't know it was broke, eh?), updates dependencies and pins some new dependencies to avoid weird conflicts in some (Leiningen) templates -- I think that's due to the difference between tools.deps and lein/`boot` in terms of resolving conflicts (gotta love those pesky Jackson libraries!).#2019-08-2618:34seancorfieldseancorfield/depstar 0.3.2 -- encountering an unknown file type in a dependency no longer prevents building a JAR (you'll either get a warning or not, depending on whether the name of the file is on the "excluded" list -- you can see more of what is going on either via the -v option or by enabling debugging).#2019-08-2716:04nathanmarztrying to use a git coordinate on a repo that has .clj and .java source files, but trying to use a namespace from the dependency in a repl fails because the java source isn't being compiled#2019-08-2716:04nathanmarzis there a way to handle this?#2019-08-2716:10alexmillerWith deps, no#2019-08-2716:10alexmillerFor projects that require a build step, you’ll need to use an artifact#2019-08-2716:10hiredmantools.deps doesn't provide any built in way to run a build step (like calling javac), I think when it has come up in the past people have talked about calling out to javac, but I am not sure if anyone has wrapped that up into something re-usable#2019-08-2716:13hiredmanand I wouldn't hold my breath for something that solves that in the general case, because you would really need something that knows how to build arbitrary java projects, much easier to rely on built artifacts#2019-08-2818:28onetomI'm wondering how can I test running clj for the 1st time in a deps.edn project, without rm -r ~/.m2
I remember in the past I could just do something like env M2_HOME=$(realpath .)/m2 boot repl or something like that, but now this is the only way I found to affect that path:
mvn -Dmaven.repo.local=$(realpath .)/m2 help:evaluate -Dexpression=settings.localRepository
but if I try something similar with clj, it still seem to use the default location (since it's not downloading anything):
rm -r .cpcache; time clj -J-Dmaven.repo.local=$(realpath .)/m2 -e 1
#2019-08-2818:37onetomAnd, btw, I'm still seeing abysmal performance when accessing Maven Central from Hong Kong.
After rm -r ~/.m2, it took 9minutes to download 25MB of jars (just clojure, netty, aleph, compojure).
While my connection is shit and hardly ever goes over 2MByte/s and it's below 1MB/s mostly, 9minutes is unreasonable.
It seems to me that the main reason is the number of requests, which are also made serially (and might not even use keep-alive?)
So I don't think only S3 is slow.#2019-08-2818:40onetomI've also checked the related DNS chain:
. 1735 IN CNAME .
. 21599 IN CNAME .
. 29 IN A 151.101.196.215
and that IP points to San Jose.#2019-08-2818:41onetomI had similar issues with the nixos binary caches. When they switched from AWS CloudFront to Fastly (which is significantly cheaper, plus they actually sponsor them), then the performance from Hong Kong dropped drastically.#2019-08-2818:42alexmilleryou can set :mvn/local-repo in your deps.edn#2019-08-2818:42alexmillerto use a different (presumably empty) local repo dir#2019-08-2818:42onetomso not from the command line or environment?#2019-08-2818:42alexmilleryou can clj -Sdeps '{:mvn/local-repo "foo"}'#2019-08-2818:42alexmillerso that's command line#2019-08-2818:43onetombut im not talking about specifying dependencies#2019-08-2818:43onetomim talking about specifying the maven cache directory#2019-08-2818:43alexmillerthe local repo is the maven cache directory#2019-08-2818:43alexmillerusually it's ~/.m2/repository#2019-08-2818:44onetomindeed, it works#2019-08-2818:44alexmillerare you using latest clj?#2019-08-2818:44alexmiller1.10.1.469#2019-08-2818:44onetomwhere is this documented?#2019-08-2818:44alexmillerclj -Sverbose will tell you#2019-08-2818:46alexmillerwe are making use of maven session caches as of that version, so would be interested in whether that makes a difference for you#2019-08-2818:46onetomversion = 1.10.1.466#2019-08-2818:46alexmillerit should mostly cut down on repeated download of metadata files though, which are pretty tiny#2019-08-2818:47onetomit would make sense to use a more common option for this, like -v; i keep forgetting it 😕#2019-08-2818:47onetomthanks, i will give it a go#2019-08-2818:47alexmillerjars are serially downloaded#2019-08-2818:49alexmillerthere are ways to change that, but, it's somewhat involved#2019-08-2818:50onetomis there an off the shelf solution for setting up a maven repo manager specifically for clojure usage?
the https://maven.apache.org/repository-management.html page mentions a lot of options.
a few years ago i've tried a few of them but i was struggling to make them work.
it should be practically a caching http proxy...
i was really hoping to find some small, turn-key solution, which i would run in the cloud, so i can share it between the office an home#2019-08-2818:51alexmillerthere are several turn-key products for this, not sure if any qualify as "small"#2019-08-2818:52alexmillerI've used Nexus a lot#2019-08-2818:52alexmillerbut there is also JFrog and Artifactory#2019-08-2818:53alexmillersorry, those are the same#2019-08-2818:53alexmillerArtifactory is by JFrog#2019-08-2818:53hiredmanyou can just use a caching http proxy too, you just don't get the nice management interfaces, and the ability to merge sources, etc#2019-08-2818:54onetomlatest nixpkgs contains 1.10.1.469 on its master branch. downloading it now:
uhu:multiboxx onetom$ nix-shell -I nixpkgs=~/nixpkgs/ -p clojure
these paths will be fetched (307.02 MiB download, 990.58 MiB unpacked):
/nix/store/0d8mpzq2dah05xqd6i1c9g02blvsvcnj-bash-interactive-4.4-p23-man
looks like it will take about 10minutes.
i will try a clean download of the same 25MB dependencies afterwards#2019-08-2818:54alexmillerthere are some benefits to using a maven-aware product I think#2019-08-2818:55alexmillerI guess one question I have is why you care what the perf is if you do it just once and then cache it forever?#2019-08-2818:55onetom@hiredman and what would you recommend as a caching http proxy? would it work with https? so for example datomic can be cached too?#2019-08-2818:56onetom@alexmiller these things never happen only once.
i have multiple personal computers, multiple office computers, more and more servers and hoping to add more colleagues too.
these caches should be primed on all those filesystems and having a bad 1st experience doesn't help...#2019-08-2818:56hiredmanI think it could be done, but it is a trade off, if you really want something a lean caching solution a caching proxy would be that, but you will have to invest time getting it working#2019-08-2818:57onetomalso on CI systems it's good to be able to build stuff afresh from time to time at least#2019-08-2818:57alexmillerI'm not trying to be flip about it, obviously perf matters, just trying to probe a little closer#2019-08-2818:58alexmillerthe structure of maven is designed that you don't need to refresh - these are immutable, uniquely versioned artifacts#2019-08-2818:58alexmillerCI is one use case where we see this come up#2019-08-2818:58onetommore specifically, i have 2 clojure projects im working on at the moment and i just wanted to know how much are their dependencies#2019-08-2818:58hiredmanat my last job we had a nexus setup that aggregated a number of different repos, and our builds were setup to only check our nexus, and that worked pretty well#2019-08-2818:58alexmillerbut most CI allow you to keep a maven cache alive#2019-08-2818:59onetomi could have built an uber jar, but my knowledge is outdated on how to do that with t.d
im just getting back to using clojure after a ~2yr break (when i was working with ethereum and js...)#2019-08-2818:59seancorfieldhttps://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging for making uberjars with t.d.a.#2019-08-2818:59alexmillerthere might also be better maven central mirrors you could use from HK, not sure#2019-08-2819:00alexmillerclj (latest) supports Maven mirrors#2019-08-2819:01onetomi looked into the mirrors but most of them doesn't work.
there is a UK one which is not offline but to do a performance test, i wanted to know how can i relocate the m2 cache, since it's not fun to delete my ~/.m2 directory 🙂#2019-08-2819:01alexmillerthere's one on google storage I know#2019-08-2819:02alexmiller<settings>
<mirrors>
<mirror>
<id>google-maven-central</id>
<name>Google Maven Central</name>
<url>https://maven-central.storage.googleapis.com</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
</settings>#2019-08-2819:02onetomnever seen that being mentioned anywhere#2019-08-2819:03alexmillerhttps://cloudplatform.googleblog.com/2015/11/faster-builds-for-Java-developers-with-Maven-Central-mirror.html#2019-08-2819:03alexmillerlooks like they have an asia-pacific one there too#2019-08-2819:06onetomi was googling for hong kong maven central proxy and came across these:
• https://repo.maven.apache.org/maven2/.meta/repository-metadata.xml
• https://maven.apache.org/guides/mini/guide-mirror-settings.html
• and one more which was also listing ibiblio.{net,org}, which are down#2019-08-2819:07onetomthanks a lot, @alexmiller !
if these work, it will really help with years of frustration 🙂#2019-08-2819:08alexmillerplease report back, would be very interested to hear if those are any better#2019-08-2819:12onetomhmm... it took one minute to download 18MB, but only the 1st dependency was printed
[nix-shell:/Volumes/Data/lab/multiboxx]$ clj -Sverbose
version = 1.10.1.469
install_dir = /nix/store/0k1f3llrx4aggbgx7rhh70mrardqvrgz-clojure-1.10.1.469-prefix
config_dir = /Users/onetom/.clojure
config_paths = /nix/store/0k1f3llrx4aggbgx7rhh70mrardqvrgz-clojure-1.10.1.469-prefix/deps.edn /Users/onetom/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1871601414.cp
Refreshing classpath
[nix-shell:/Volumes/Data/lab/multiboxx]$ time clj -e 1
Downloading: riddley/riddley/0.1.12/riddley-0.1.12.pom from
1
real 1m18.496s
user 0m13.404s
sys 0m0.742s
[nix-shell:/Volumes/Data/lab/multiboxx]$ ls m2/
cheshire/ clj-tuple/ commons-io/ me/ riddley/ tigris/
clj-http/ com/ commons-logging/ org/ seancorfield/
clj-soup/ commons-codec/ etaoin/ potemkin/ slingshot/
#2019-08-2819:13onetomah, nvm, i forgot to rm -r m2 .cpcache beforehand#2019-08-2819:17onetom2nd run is printing all the downloaded deps:
[nix-shell:/Volumes/Data/lab/multiboxx]$ rm -r m2 .cpcache/; time clj -e 1; du -hsc m2
Downloading: org/clojure/clojure/1.10.1/clojure-1.10.1.pom from
...
Downloading: org/apache/httpcomponents/httpmime/4.5.2/httpmime-4.5.2.jar from
Downloading: com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.7.5/jackson-dataformat-smile-2.7.5.jar from
Downloading: org/clojure/data.codec/0.1.0/data.codec-0.1.0.jar from
1
real 1m30.756s
user 0m16.289s
sys 0m1.102s
18M m2
18M total
this is magnitudes better!
i guess i should make a counter test with the old version just to confirm im not just experiencing some change in network conditions#2019-08-2819:18alexmillerusing -Sforce will let you force a classpath recompute (don't need to nuke your .cpcache then)#2019-08-2819:19alexmilleris that difference with just the new clj version or the new clj version + mirror?#2019-08-2819:21onetomonly the clj version is different#2019-08-2819:21onetomthis is my deps.edn:
{:paths ["src" "rsc"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/data.csv {:mvn/version "0.1.4"}
clj-soup/clojure-soup {:mvn/version "0.1.3"}
me.raynes/fs {:mvn/version "1.4.6"}
org.clojure/java.jdbc {:mvn/version "0.7.9"}
org.xerial/sqlite-jdbc {:mvn/version "3.28.0"}
com.microsoft.sqlserver/mssql-jdbc {:mvn/version "7.2.2.jre8"}
seancorfield/next.jdbc {:mvn/version "1.0.5"}
etaoin {:mvn/version "0.3.5"}}
:aliases {:test {:extra-paths ["test"]}}
:mvn/local-repo "m2"
:mvn/repos {
"central" {:url ""}
;"uk" {:url ""}
"clojars" {:url ""}
}}
#2019-08-2819:22onetomnext thing after the old version finishes is to try the google mirror#2019-08-2819:23onetomwhich i guess, i can simply do by replacing the :url for the "central" entry to the google mirror's url, right?#2019-08-2819:23onetombut currently im seeing 7-30kbyte/s download rates with the old version, so it will take awhile...#2019-08-2819:24onetomnope, it took only ~3mins:
uhu:multiboxx onetom$ clojure -Sverbose
version = 1.10.1.466
install_dir = /nix/store/m2v1xd7cybi544jyl77w3mzjw1xklw41-clojure-1.10.1.466-prefix
config_dir = /Users/onetom/.clojure
config_paths = /nix/store/m2v1xd7cybi544jyl77w3mzjw1xklw41-clojure-1.10.1.466-prefix/deps.edn /Users/onetom/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1871601414.cp
Refreshing classpath
^Cuhu:multiboxx onetom$ rm -r m2 .cpcache/; time clj -e 1; du -hsc m2
rm: cannot remove '.cpcache/': No such file or directory
...
Downloading: org/clojure/data.codec/0.1.0/data.codec-0.1.0.jar from
1
real 3m21.764s
user 0m18.844s
sys 0m1.207s
18M m2
18M total
#2019-08-2819:25onetomstill, it's quite significant difference, so thanks a lot for telling me about this enhancement and for implementing these improvements!#2019-08-2819:29onetomclj 1.10.1.466
mirror: https://maven-central-asia.storage-download.googleapis.com/repos/central/data/
uhu:multiboxx onetom$ rm -r m2; time clj -Sforce -e 1; du -hsc m2
...
real 1m40.781s
user 0m21.119s
sys 0m1.290s
18M m2
18M total
#2019-08-2819:32onetom2/3 of the dependencies come from maven central and the rest is clojars:
uhu:multiboxx onetom$ rg -c maven-central-asia 466-deps.txt
71
uhu:multiboxx onetom$ rg -c 466-deps.txt
22
uhu:multiboxx onetom$ wc -l 466-deps.txt
92 466-deps.txt
(it almost adds up :)#2019-08-2819:35onetomclj 1.10.1.469
mirror: https://maven-central-asia.storage-download.googleapis.com/repos/central/data/
uhu:multiboxx onetom$ rm -r m2; time clj -Sforce -e 1; du -hsc m2
...
real 0m44.863s
user 0m13.960s
sys 0m1.059s
18M m2
18M total
#2019-08-2819:35onetomthat consistently seems to be a more than 2x difference!#2019-08-2819:36alexmillerpretty big combined change! :)#2019-08-2819:36onetomalthough it's possible that im warming up some CDN caches...#2019-08-2819:37onetomdo you happen to know any asia-pacific clojars mirror by any chance? 🙂#2019-08-2819:37onetomi will try this mirror on my other project too#2019-08-2819:39alexmillerI don't believe there is anything region specific. they do have a cdn mirror though#2019-08-2819:39alexmillerI have no idea of difference in perf though#2019-08-2819:48onetomthis google mirror understands HTTP2:
uhu:reap onetom$ curl -vso /dev/null --http2 2>&1 | rg -i http2
* Using HTTP2, server supports multi-use
is the HTTP client in clj talking HTTP2?#2019-08-2819:49alexmillerthat's several layers below the code I'm in, so don't know#2019-08-2819:49alexmillerit would not surprise me if the answer was no#2019-08-2819:53alexmillerthe maven resolver uses org.apache.httpcomponents/httpclient and the version it's pinned on does not look like it supports http 2 (the newest series does though)#2019-08-2819:54alexmillerso my strong guess would be: no, but it potentially could#2019-08-2819:59onetomin my other project; 24MB of deps:
clj 1.10.1.469
mirror: https://maven-central-asia.storage-download.googleapis.com/repos/central/data/
[nix-shell:~/lab/reap]$ rm -r m2; time clj -Sforce -e 1; du -hsc m2
...
real 2m53.788s
user 0m24.007s
sys 0m1.858s
24M m2
24M total
#2019-08-2820:06onetomclj 1.10.1.466
mirror: https://maven-central-asia.storage-download.googleapis.com/repos/central/data/
[nix-shell:~/lab/reap]$ rm -r m2; time clj -Sforce -e 1; du -hsc m2
...
real 3m33.490s
user 0m26.609s
sys 0m1.819s
24M m2
24M total
that's less of a difference, but still quite noticable (~20%)
anyway, enough testing for tonight; it's 4am in HK, so i shall sleep.
thanks again for your attention and advice!#2019-08-2904:33onetomit happened again.
when i've added the :mvn/local-repo "m2" to my deps.edn and ran time clj -e 1 for the 1st time, i've only seen some of the dependencies being printed, while it was actually downloading all of them:
Downloading: ring/ring-jetty-adapter/1.6.3/ring-jetty-adapter-1.6.3.pom from
Downloading: crypto-equality/crypto-equality/1.0.0/crypto-equality-1.0.0.jar from
Downloading: potemkin/potemkin/0.4.5/potemkin-0.4.5.jar from
1
real 1m42.094s
user 0m16.879s
sys 0m0.933s
raven:reap onetom$ du -hsc m2
22M m2
22M total
#2019-08-2904:37onetomafter an rm -r m2, on subsequent run, i could see all of the deps printed as they were being downloaded and the size of the maven cache ended up being the same:
raven:reap onetom$ rm -r m2
raven:reap onetom$ time clj -Sforce -e 1
Downloading: org/clojure/data.json/0.2.6/data.json-0.2.6.pom from
...
Downloading: com/google/code/gson/gson/2.7/gson-2.7.jar from
1
real 0m38.062s
user 0m16.855s
sys 0m1.167s
raven:reap onetom$ du -hsc m2
22M m2
22M total
something, somewhere might be caching these artefacts, because the 2nd time was so much faster...
but MAYBE the first time is slower because of the same bug, which causes only a few of the downloaded dependencies being printed!#2019-08-2904:38onetomfor the record, my deps.edn looks like this:
{:paths ["src" "rsc" "java" "classes"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
clj-http {:mvn/version "3.10.0"}
cheshire {:mvn/version "5.1.1"}
org.clojure/data.json {:mvn/version "0.2.6"}
com.squareup.wire/wire-runtime {:mvn/version "3.0.0-rc01"}
com.google.protobuf/protobuf-java {:mvn/version "3.9.1"}
com.google.protobuf/protobuf-java-util {:mvn/version "3.9.1"}
zprint {:mvn/version "0.4.16"}
ring/ring-core {:mvn/version "1.7.1"}
ring/ring-devel {:mvn/version "1.7.1"}
ring/ring-jetty-adapter {:mvn/version "1.6.3"}
aleph {:mvn/version "0.4.6"}
compojure {:mvn/version "1.6.1"}
hiccup {:mvn/version "1.0.5"}
;org.clojure/clojurescript {:mvn/version "1.10.520"}
}
:aliases {:test {:extra-paths ["test"]
:extra-deps {
;clj-http-fake {:mvn/version "1.0.3"}
}}}
:mvn/local-repo "m2"
;
:mvn/repos {"central" {:url
""
#_""}
"clojars" {:url ""}}}
#2019-08-2904:39onetomand my clj version info looks like this:
raven:reap onetom$ time clj -Sverbose -e 1
version = 1.10.1.469
install_dir = /nix/store/bqs5l2mc2jlj5z1a21ix0sz0ak8cvn0l-clojure-1.10.1.469-prefix
config_dir = /Users/onetom/.clojure
config_paths = /nix/store/bqs5l2mc2jlj5z1a21ix0sz0ak8cvn0l-clojure-1.10.1.469-prefix/deps.edn /Users/onetom/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1871601414.cp
1
real 0m0.858s
user 0m2.045s
sys 0m0.131s
#2019-08-2904:39onetom/Users/onetom/.clojure/deps.edn is all commented out#2019-08-2913:35hoynkIs it possible to just download the dependencies of my project, without running anything? Like with lein deps.#2019-08-2913:36dominicmDoing -Spath will print the classpath, and won't execute anything.#2019-08-2913:37hoynkNice trick, thanks!#2019-08-3021:26andy.fingerhutFYI, I have used deps.edn to start both Clojure/Java and ClojureScript/Node.js REPLs, and run tests for each, for the core.rrb-vector that I have been experimenting with some proposed bug fixes for, here: https://github.com/jafingerhut/core.rrb-vector/blob/proposed-fixes-for-4-issues/deps.edn#2019-08-3021:27andy.fingerhutIt basically uses a bunch of aliases, some prefixed with ":clj-" and others with ":cljs-", for similar functionality in each Clojure flavor. The best alternative I could imagine to that approach would be to have two different directories with different deps.edn files, one for Clojure/Java, the other for ClojureScript.#2019-08-3021:29andy.fingerhutCertainly open to other ideas if anyone thinks of improvements to that approach.#2019-08-3021:33Felipe Cortezshould add-lib work for every lib or are there limitations?
(require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
(add-lib 'cheshire {:mvn/version "5.9.0"})
(require '[cheshire.core :as json])
fails with
Syntax error (ClassNotFoundException) compiling new at (cheshire/factory.clj:57:11).
com.fasterxml.jackson.core.async.ByteArrayFeeder
Syntax error compiling at (cheshire/core.clj:1:1).
namespace 'cheshire.factory' not found
#2019-08-3021:36dominicmDoes it work if you include it in your deps.edn?#2019-08-3021:36Felipe Cortezyep#2019-08-3021:47seancorfieldI ran into something like this with clj-new and the Luminus template... as I recall it's a peculiarity of the Jackson library as far as its dependencies go...#2019-08-3021:47seancorfield(or maybe it's a bug in how Cheshire is packaged?)#2019-08-3021:47seancorfieldI ended up needing to explicitly depend on various Jackson libraries in order to get that to succeed...#2019-08-3021:50alexmilleradd-lib will interpret new libraries in the context of your existing libs#2019-08-3021:50alexmillerso if you already have an existing version of jackson on the classpath, and you add cheshire, it (can't) override the versions you already have#2019-08-3021:51alexmillerif cheshire needs a newer version than the one you have, it may not work#2019-08-3021:51alexmillerwhereas if you had run a deps resolution on the full set of deps, it might have been able to resolve to a better set of deps#2019-08-3021:52alexmillerjackson tends to be involved in 75% of dependency conflicts, so it's always a likely culprit#2019-08-3021:52seancorfieldYeah, that sounds like the problem -- Cheshire/Jackson version mismatches.#2019-08-3021:54alexmillerthis is a totally normal thing that can happen with add-libs#2019-08-3021:54seancorfieldS3 wagon lib pulls in Jackson 2.5.5#2019-08-3021:56Felipe Cortezmakes sense, thanks!#2019-08-3021:57seancorfield@clojurians884 Here's a combination that works: (! 508)-> clj -A:deps -Sdeps '{:deps {com.fasterxml.jackson.core/jackson-core {:mvn/version "2.7.5"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'cheshire {:mvn/version "5.6.3"})
true
user=> (require 'cheshire.core)
nil
user=>
#2019-08-3021:57seancorfieldNote the Jackson override on the command line. Without that, it won't work.#2019-08-3021:58Felipe Cortezawesome! thanks for the help 🙂#2019-08-3021:58seancorfieldYou don't need to add-lib on the smile sub-lib either (edited to remove that)#2019-08-3021:59seancorfieldIf you want to use the latest Cheshire, you need Jackson 2.9.9: (! 510)-> clj -A:deps -Sdeps '{:deps {com.fasterxml.jackson.core/jackson-core {:mvn/version "2.9.9"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'cheshire {:mvn/version "RELEASE"})
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
true
user=> (require 'cheshire.core)
nil
user=>
#2019-08-3022:00seancorfield(I would love to excise Cheshire and Jackson from our code base for exactly this reason!)#2019-08-3022:01seancorfield(and the problem is because tools.deps brings in s3-wagon-private which in turn brings in Jackson 2.5.5 so you need an explicit override to make it all work)#2019-08-3022:01Felipe Cortezdo you recommend using data.json instead?#2019-08-3022:04alexmillerI've got a spike of some work to use the cognitect s3 lib completely in place of the s3 wagon for tools.deps, would love to shift things over, but needs some work still (and prob streaming support in the s3 lib)#2019-08-3022:05alexmillerI think the cognitect stuff uses data.json, which is definitely not as fast as cheshire/jackson (but way less problematic)#2019-08-3022:07seancorfieldIf data.json does what you need -- and does it fast enough -- then, yes, use it instead of Cheshire to avoid the pit of dependency hell that is Jackson 🙂 But, in reality, you'll find it hard to avoid Cheshire as other libraries bring it in too...#2019-08-3022:10seancorfield(having said that, I'm now having a hard time finding a single library that depends on Cheshire! oh how I wish that http://cross-clj.info was still alive!)#2019-08-3022:11seancorfield(it was without - I think but that brings up an online casino that is squatting on the domain!)#2019-08-3022:13Felipe Cortezagain, thanks for the help 😄#2019-08-3103:59sogaiuis the purpose of -Dclojure.libfile=... in the invocation at the end of the clojure bash (powershell) script diagnostic / for debugging?#2019-08-3104:36seancorfieldI'm not seeing anything in the code at first glance that uses it so... I don't know... but it's a good question to ask Alex when he's around (Tuesday, probably, given that it's Labor Day weekend here) @sogaiu#2019-08-3104:40sogaiu@seancorfield tnx!#2019-08-3110:53alexmillerIt’s to communicate for add-lib or tools like it#2019-08-3111:20sogaiu@alexmiller thanks for the explanation -- i presume it's not considered an api one can rely on (i.e. the ability to access the info via the clojure.libfile system property), is that right?#2019-08-3111:25alexmillerIt has to be considered optional. The impl in add-lib knows how to use several means to detect the lib set, including classpath introspection#2019-08-3111:26alexmillerThat whole thing is potentially something we may add to Clojure itself, tbd (which is why add-lib is not final)#2019-08-3111:35cflemingI’d really like to use Deps as a lightweight way to manage deps for projects which aren’t always Clojure, or all Clojure. Currently there’s no way to globally exclude Clojure itself from the dependencies, correct?#2019-08-3111:36cflemingAs I understand it, in current Deps versions the system deps.edn is always read from the bundled copy in t.d.a, and there’s also no way to override that - is that correct?#2019-08-3111:38cflemingI did try this in my deps.edn, but it didn’t work: {org.clojure/clojure {:mvn/version "1.10.1" :exclusions [org.clojure/clojure]}}#2019-08-3111:40cflemingWould there be interest in a patch adding global exclusions support to deps, say using a top-level :exclusions key?#2019-08-3111:44cflemingI also tried: {org.clojure/clojure {:local/root "empty"}} where empty/deps.edn is just {}, also no dice.#2019-08-3114:50kennyGlobal exclusions would be very useful for us too. #2019-08-3115:29alexmillerthere is currently no way to "remove" a dependency from the top level, and clojure is included as a default dependency, so the practical effect is that clojure is always included#2019-08-3115:29alexmillerI don't consider this to be a major problem as the in-scope use case here is running Clojure programs (I don't consider clj to be a tool for generic deps mgmt)#2019-08-3115:31alexmillerthat said, we do have a ticket for global exclusions and there certainly are times when that would be useful. similarly, it's pretty hard to use a different dep type (like local or git) to override the clojure version (due to issues in overrides of different dep type and version comparison), and that's something that I sometimes need to do#2019-08-3115:33alexmilleranyhow, I'm still thinking about the best solution to these#2019-08-3120:46cflemingAnother use case I have is running a Clojure fork.#2019-08-3121:08alexmillerYeah, that’s unintentionally difficult as I discovered recently when I was trying to do something similar#2019-09-0119:37andy.fingerhutIf I run clj -Sdep '{:deps {org.clojure/clojure {:git/url "" :sha "...."}}}' I get Error: Could not find or load main class clojure.main. Is there a way to run arbitrary git commits of Clojure using clj command?#2019-09-0119:38andy.fingerhutI had an actual SHA hex string instead of ..... there, of cousre.#2019-09-0119:43andy.fingerhutThis is with version 1.10.1.469 of clj on Linux#2019-09-0119:44andy.fingerhutWait, I guess that cannot work, since it needs to compile Java code. Never mind.#2019-09-0215:48drewverleewhats a short summery of what the .cpcache folder contains?#2019-09-0216:28alexmiller.cp files (final classpaths) and .lib files (intermediate dep files)#2019-09-0216:29alexmillerclj either uses the cached cp file or has to spin a separate jvm with tools.deps to compute the classpath#2019-09-0216:30alexmillerIt is truly a cache (key is a hash of a bunch of stuff) so it’s safe to rm it#2019-09-0216:49drewverleemakes sense, thanks for the explanation Alex.#2019-09-0315:46kennyHas anyone tried compiling the tools-deps with graal to increase performance?#2019-09-0315:48alexmillerthere's really two stages in clj and the first one (cp computation) should be pretty amenable to it#2019-09-0315:48alexmillerit makes the install story (which is already complicated) much more complicated#2019-09-0315:49alexmillerand in most cases the result of cp computation is a) IO-bound and b) cached and skipped#2019-09-0315:49alexmillergiven all that, exploring this is not a high priority for me#2019-09-0316:05kennyPrimarily interested in the first one#2019-09-0317:44WhittlesJrHi! I think I asked a while back about using self-signed certs for accessing a self-hosted maven repo over https with tools.deps. (The Leiningen equivalent is adding the path to the cert in :certificates [].) IIRC, at the time that I asked, this was not supported with tools.deps. I can't seem to find anything about this by googling or looking on JIRA. Could anyone give me an update on whether this works these days, or point me in the right direction for how to handle this scenario?
Relevant config in deps.edn:
:mvn/repos {"central" {:url "https://<my-nexus-server>/repository/maven-public/"}}
Everything works fine over http with tools.deps, and even with https with lein. But with https on tools.deps I get "unable to find valid certification path to requested target."#2019-09-0318:07alexmillerNothing has been added in this area#2019-09-0318:08alexmillerCould add it to Java’s certs?#2019-09-0318:17WhittlesJrThat's a good solution on most OSs, but I'm on NixOS which aims to be "immutable" so the keystore is read-only. I should probably look into adding the ability to add certs to the java keystore declaratively in NixOS.
Anyway, thanks for the update!#2019-09-0317:59Adrian SmithI'm struggling to import a library that I've forked on my own repo details: https://ask.clojure.org/index.php/8507/failure-on-github-import-via-deps#2019-09-0318:00WhittlesJr@sfyire I'm pretty sure that implies that the repo in question does not have a deps.edn (which it doesn't). To my knowledge, if you want to use :git/url, the target must have a deps.edn.#2019-09-0318:01Adrian Smith@alex.joseph.whitt ah thank you#2019-09-0318:03alexmillerYou can tell it to do that#2019-09-0318:05Adrian Smith@alexmiller how would I do that?#2019-09-0318:05alexmillerIf you add :deps/manifest :deps to the coord that overrides the autodetection (although you won’t get the deps in project.clj)#2019-09-0318:06alexmillerYou could add those to the caller though if you don’t have access to modify the project#2019-09-0318:06alexmillerIf you’ve forked it, then I’d just add a deps.edn#2019-09-0318:06Adrian Smithyeah, that's cool to know though#2019-09-0318:07WhittlesJrTIL#2019-09-0318:17seancorfield(I added the above as an answer on the ask site)#2019-09-0318:25Adrian Smiththank you#2019-09-0522:30QuestWhat's the equivalent of lein install for a clj deps.edn library?#2019-09-0522:32Quest(I don't have any packaging library right now -- just looking through the options at https://github.com/clojure/tools.deps.alpha/wiki/Tools for something easy to get running)#2019-09-0522:37andy.fingerhutIf you mean creating a JAR file of Clojure source code, I believe depstar is a good choice.#2019-09-0522:50QuestAppreciate the recommendation, easy enough to get a jar generated with clojure -A:depstar -m hf.depstar.jar northstar-model-0.0.1.jar.
The consume the library artifact from other local projects, I take it I manually install to my ~/.m2/ from here -- i.e. copying pom.xml & northstar-model-0.0.1.jar to ~/.m2/northstar/model/0.0.1/#2019-09-0523:05seancorfield@quest If you have mvn installed, that's probably the easiest way to do it "right".#2019-09-0523:06seancorfield(`mvn` is what I use to deploy JARs built with depstar to http://clojars.org -- I've never tried to to install locally but I'm pretty sure it's a similar command)#2019-09-0523:09seancorfieldmvn install:install-file -Dfile=northstar-model-0.0.1.jar -DpomFile=pom.xml
I believe#2019-09-0523:10seancorfield#2019-09-0523:14QuestHmm, I'll keep looking through maven install & see if I can find something that works. Faking the .m2 autogen files isn't working out#2019-09-0523:15seancorfield@quest The above command seemed to work for me, based on limited testing#2019-09-0523:17QuestYes! I can confirm that it's working. Thanks @seancorfield
clojure -A:depstar -m hf.depstar.jar northstar-model-0.0.1.jar
mvn install:install-file -Dfile=northstar-model-0.0.1.jar -DpomFile=pom.xml
;; include dependency [northstar/model "0.0.1"]
#2019-09-0523:17QuestIt seems like it'd be good to document this somewhere, but I'm not exactly sure where 🤔#2019-09-0523:18seancorfieldI should add it to the readme...#2019-09-0523:18QuestI'll open a PR with my best attempt to capture it, edit it as needed from there? 🙂#2019-09-0523:19seancorfieldSure! Thank you!#2019-09-0523:19seancorfieldDeploy to Clojars is mvn deploy:deploy-file -Dfile=${1}.jar -DpomFile=pom.xml -DrepositoryId=clojars -Durl=#2019-09-0523:20seancorfield(where ${1} is the jar file name -- and this assumes your credentials are in ~/.m2/settings.xml like <settings>
<servers>
<server>
<id>clojars</id>
<username>someperson</username>
<password>topsecret</password>
</server>
</servers>
</settings>
#2019-09-0523:45Questopened https://github.com/seancorfield/depstar/pull/17#2019-09-0615:43TravisI may have over looked this in the docs but is there something kind of the equivalent of lein deps
. Basically trying to make sure all maven deps are downloaded for caching#2019-09-0615:55seancorfield@camechis use clojure -Spath or clojure -Stree (and just ignore the output).#2019-09-0615:56Travissweet, thnx @seancorfield#2019-09-0616:49jjttjj@quest this might not fit your use case but if you're just looking to use code from one of your local projects it's dead easy to just refer to local source directories with clj: :deps {
...
{:local/root "/home/jjttjjj/code/goodlib"}
...
}#2019-09-0616:50alexmilleryou should use :paths for that#2019-09-0617:50Quest@jjttjj Unfortunately my other projects aren't deps.edn yet -- adapting the re-frame 10x (shadow-cljs) template isn't something I'm ready for yet. Will keep that in mind though#2019-09-0702:15seancorfieldseancorfield/depstar "0.3.3" is available -- no longer uses clojure.xml so you won't get "Illegal Reflective Access" warnings on JDK11 any more!#2019-09-0718:32tianshuI'm trying to create my own clj project template using clj-new. when using it, there's an error:
Failed with: Failed to resolve version for :home/tianshu/develop/ministack/lein-template:jar:RELEASE: Could not find metadata :home/tianshu/develop/ministack/lein-template/maven-metadata.xml in local (/home/tianshu/.m2/repository)
Execution error (ExceptionInfo) at clj-new.helpers/resolve-remote-template (helpers.clj:148).
Could not load artifact for template: /home/tianshu/develop/ministack
Tried coordinates:
[/home/tianshu/develop/ministack/boot-template "RELEASE"]
[/home/tianshu/develop/ministack/lein-template "RELEASE"]
Full report at:
/tmp/clojure-14768781517611096608.edn
what does this mean, why a lein-template and boot-template is needed?#2019-09-0718:36hiredmanyou are somehow declaring a maven dependency with a group-id of /home/tianshu/develop/ministack/ and an artifact-id of lein-template#2019-09-0718:36hiredmanwhich is obviously bogus#2019-09-0718:38tianshuBut it is said I should use qualified symbol or multi-segment clojure symbols.
Project names must be valid qualified or multi-segment Clojure symbols.#2019-09-0718:38tianshuOhh#2019-09-0718:38tianshuI know#2019-09-0718:38tianshusorry#2019-09-0718:40tianshuI give it the wrong template path#2019-09-0718:41seancorfieldclj-new looks for a clj template, then a boot template, then a lein template -- so you can use it to create projects with other tools' templates.#2019-09-0718:42seancorfield(it's based on boot-new which looks for a boot template, then a lein template -- and I based that library in turn on the innards of lein new, and in fact both boot-new and clj-new include some Leiningen code, with permission)#2019-09-0718:47tianshuthanks for the explain!#2019-09-0718:48seancorfieldThe idea behind "Project names must be valid qualified or multi-segment Clojure symbols." is to avoid the everything.core default that Leiningen (and Boot) fall back on if you provide a single segment name.#2019-09-0718:52seancorfieldFor qualified names, it sort of expects you to use something relatively unique like your GitHub username when creating a project: clj -A:new some-template tianshu/example which will create a folder called example but the main namespace in that project would be tianshu.example#2019-09-0718:53seancorfieldOf course you can just go ahead and produce the same thing as Leiningen: clj -A:new some-template example.core 🙂 The folder will be example.core but the main ns will be example.core from src/example/core.clj ... ugh!#2019-09-0718:55seancorfieldAnd finally if you combine them: clj -A:new app tianshu/example.core you'll get a top-level example.core project folder with src/tianshu/example/core.clj for a main ns of tianshu.example.core#2019-09-0807:45tianshuyeah, make sense. thanks very much! @seancorfield#2019-09-0822:10cflemingI’m adding code to Cursive to cache project dependency resolution, and also classpaths for things like running REPLs. Here’s what I’m hashing to create a cache key:
1. Global config (whether using CLI or t.d.a directly, and identifying properties like install path & version in each case), and user and system (if applicable) deps.edn files.
2. Hash of all non-whitespace non-comment content from project deps.edn, and all transitive deps.edns reachable via :local/root.
3. Aliases in use, depending on operation.#2019-09-0822:11cflemingIn the case of the execution classpath, I’ll also add in details from the execution, e.g. main namespace, env vars etc, basically all the info from the run configuration.#2019-09-0822:11cflemingIs there anything else that might affect that I’ve forgotten?#2019-09-0822:20seancorfieldWhat about the user-level deps.edn which may include aliases/extra-deps/override-deps?#2019-09-0822:22seancorfieldAt World Singles Networks, we use CLJ_CONFIG to set the "user-level" to be our repo-wide deps.edn and then each subproject has a deps.edn file. Those two combined then form the set of deps and aliases etc for each given subproject. We use the repo-wide ("user-level") deps.edn file to "pin" versions by having a :defaults alias with all our :override-deps in.#2019-09-0900:16cfleming@seancorfield Ah, yes, thanks - I actually include that already in option #1#2019-09-0900:16cflemingAlthough Cursive doesn’t yet support CLJ_CONFIG - that’s on my TODO list#2019-09-0900:18cflemingI’ve edited the list above to make that clear.#2019-09-0900:19cflemingThose files include the same non-whitespace non-comment content logic.#2019-09-0900:23alexmillerdid you look at the clojure script? you can see everything included in the hash now#2019-09-0900:24alexmillerseems like matching that is best for matching behavior#2019-09-0900:25seancorfield@alexmiller Did you say CLJ_CONFIG support might go away? Is there an alternative in mind where a (large) project/monorepo may want a "base" deps.edn file and then to have a separate deps.edn for each subproject?#2019-09-0900:25alexmillerno, I didn't say that#2019-09-0900:25seancorfieldAh, then I perhaps misunderstood you 🙂#2019-09-0900:26alexmillermore likely to add something to insert more#2019-09-0900:26seancorfieldOK. Having four deps.edn files being considered would be fine too. System, user, (project/repo -- new!), (sub)project/current dir.#2019-09-0900:27seancorfield(I mostly don't care about the first two in this case -- I really care about the "third" and fourth 🙂 )#2019-09-0900:29cflemingI did look at the script, but a while ago. IIRC it doesn’t include deps file contents, only the config paths, and definitely not the contents of transitive local roots.#2019-09-0900:29cflemingAt least, I couldn’t see where the deps file contents were taken into account.#2019-09-0900:33alexmilleryeah it's just the paths and aliases and the -Sdeps contents#2019-09-0900:34cflemingActually, does the -nt test take the file timestamps into account too? My google-bash-fu is failing me.#2019-09-0900:34alexmiller"newer than"#2019-09-0900:35cflemingRight, that was what I suspected, but Google wasn’t very helpful.#2019-09-0900:35alexmillerso the "stale" calculation is separate from the hash#2019-09-0900:35cflemingRight.#2019-09-0900:36cflemingIn my case I don’t need a location (i.e. what you’re using the hash for), I just need the staleness test.#2019-09-0900:36alexmillerso yeah, that's largely timestamp based#2019-09-0900:37cflemingOk, thanks. In my case it’s relatively easy to do the full transitive contents check, so I’ll stick with that.#2019-09-0900:37alexmillerwell, and assuming the hash finds there is a cp to use#2019-09-0900:38cflemingRight.#2019-09-0900:43cflemingActually, one other question I had: since the system deps file is now read from a resource, am I correct in thinking that for recent versions I only need to pass the user config, and not both?#2019-09-0900:44cflemingI looked at the code a while back, and it looked like the resource will always be put first in the list, so if I also pass the system deps file then that will override the one from the resource, but will presumably be the same?#2019-09-0900:45cflemingIs there any way to tell from e.g. -Sdescribe whether I need to pass the system file or not? The path still appears in the config-files list.#2019-09-0900:48cflemingIf not I can just check the version and choose based on that.#2019-09-0900:52seancorfield@cfleming Looks like you could check :install-dir against the first path in :config-files -- but I will note that t.d.a. has an API for getting the deps.edn files that returns just the user + project level ones (because it has the system one baked in).#2019-09-0901:07cflemingInteresting. It’s potentially tricky to call that API ahead of time when delegating to the CLI.#2019-09-0901:07cflemingI’ll take a look at that though, thanks.#2019-09-0902:06alexmillerit wouldn't hurt to include both#2019-09-0903:04cflemingIs there a reason that t.d.a and tools.gitlibs print progress information (“Downloading: xxx” and “Checking out: xxx”) to stderr?#2019-09-0903:20alexmillerso you can shunt it off if you don't want to see it#2019-09-0903:22alexmillernormally it's silent on stdout unless you do something that expects a printed result, like -Stree or -Spath etc#2019-09-0904:07cflemingAh, ok, thanks.#2019-09-1016:37jjttjjIs there any obvious reason I'd get the message
Skipping paths: resources
after running clj -Spom when resources is in my :paths?#2019-09-1016:38alexmillerit means it saw you had that in paths, but there is no canonical way to do that in pom.xml so, it just didn't do anything with it#2019-09-1016:39alexmillerin Maven, you typically need to pull in a plugin and do several steps to work with a resources dir#2019-09-1016:40jjttjjstrange, I somehow managed to deploy to clojars on a different computer (with depstar+maven), and don't remember any issues getting resources in the jar#2019-09-1315:48plexusI'm having some trouble using a jar from a private Maven repo (we're using Gitlab's built-in maven repos)#2019-09-1315:49plexusI can install it fine with Maven, and see it in ~/.m2, but when I do clj it seems it still tries to fetch it, Error building classpath. Could not find artifact group:artifact:jar:version in central ()#2019-09-1315:49plexusis this to be expected, or am I holding it wrong?#2019-09-1315:56dominicmI think it will try there, but it should also try from your repo that's configured#2019-09-1320:21vlaaadIf anyone is curious: I've setup a simple testing workflow using github actions and tools-deps: https://github.com/cljfx/cljfx/blob/master/.github/workflows/test.yml#2019-09-1320:25vlaaadCouple of notes:
- there is java 8 by default, so if you don't need a newer one you can omit actions/ action
- use clojure instead of clj, duh#2019-09-1320:27vlaaadI also want to say again how much I love tools-deps. Aliases and their simplicity is amazing: adding workflow-specific aliases is such a breeze, and mixing them in different contexts is very straightforward. Thank you very much cognitect and @alexmiller#2019-09-1320:30vlaaadI have :test alias to add test directory and test-specific deps, I have :runner alias to run tests, I have :headless alias to run cljfx without display. During dev I have :test alias on, I can run tests locally with -A:test:runner, and I can make CI run tests in headless mode with -A:test:runner:headless. So simple clj#2019-09-1320:31gerredhey! as someone doing a lot with github actions too, this is great. I might recommend using some more Docker-based actions rather than shell-based ones, which might make things a little bit faster.#2019-09-1320:38vlaaadBut then I'd need to figure out docker in addition to github actions, I'm too lazy 😛#2019-09-1715:23aviTotally understandable Vlad, but if and when you do want to give it a try, I think GitHub has done a good job making this super simple. Basically you just add a container key to your job definition and Actions takes care of getting the image, starting a container, and then running your job’s steps within that container.
So for example you could add container: clojure:tools-deps to your job definition and then in your steps you wouldn’t need to set up Java nor install Clojure. Presumably this would become faster over time as GitHub does a better job of caching docker image layers. And there are a bunch of variants of the clojure image, e.g. with different OS’s, different JREs, etc.
HTH but feel free to ignore if it’s not the right time!#2019-09-1715:27gerred:+1: also happy to help 🙂#2019-09-1715:29gerredit'll even build a repo from a Dockerfile, which adds time but is great for development#2019-09-1715:30aviOoo really? I didn’t know that — that’s excellent! I gotta look up the docs on that. Thanks!#2019-09-1715:30gerredof course. if you use actions/checkout and actions/docker, it's all Dockerfile based so there's pre-steps to build those images.#2019-09-1715:31gerredi'd like to optimize that, but here's an example workflow: https://github.com/kudobuilder/kudo/blob/master/.github/workflows/release-on-master.yml#2019-09-1715:31gerredactions/checkout and actions/docker will get build at the start of the workflow, which you can see here: https://github.com/kudobuilder/kudo/commit/7ee65861db6b426e87618bd44b94b469a0e99802/checks#2019-09-1715:31gerred"Build x..." aren't steps in our workflow#2019-09-1715:31gerredthey just happen#2019-09-1715:32aviAha, that makes sense! It’s kinda naturally intuitive once I unlearn the constraints I learned from CircleCI, wherein for a given job I have to choose between a VM and a container, so there’s no concise way to build an image and then use it.#2019-09-1715:32aviThanks!#2019-09-1715:33gerredof course.#2019-09-1715:34gerredi'm also well integrated with the actions team and the implementation since we're doing a lot kubernetes-wise with this, so if you need any help or support, let me know and I can either help or point you at the right people within github.#2019-09-1715:41aviWill do — thanks!#2019-09-1715:42vlaaadHey, many thanks for the info! I have a question: what problem does docker solve that actions don't? You mentioned that you can have different docker images with different OSs, JREs — isn't it something already solved by actions matrix?#2019-09-1715:43gerredactions in-and-of-themselves are either based on Docker or javascript.#2019-09-1715:43gerredworkflows are made up of a series of actions, which build up matrices, etc.#2019-09-1715:44aviI’m not sure about actions matrix; I haven’t looked closely at it yet. As for using docker in general, it can sometimes make jobs more concise and sometimes, theoretically, faster, because the images might be cached.#2019-09-1715:44gerredyou can also technically just run shell commands on the host runs-on I guess.#2019-09-1715:44gerredbrb, doing a CNCF TOC presentation, I'll be back in a bit.#2019-09-1715:44aviIt can also make it easier to debug/troubleshoot issues in a CI job because you can then use the same exact environment locally to reproduce the situation#2019-09-1715:46vlaaadAh, you mean if I have docker setup at local machine#2019-09-1715:46aviYup#2019-09-1715:46vlaaadGot it, thanks!#2019-09-1715:46gerredand you can think of an action as an individual step (but reproducible as a lego block) in a whole workflow#2019-09-1715:46gerredwith the v2 actions API#2019-09-1320:31gerred(I have a clj-kondo action currently in progress as well)#2019-09-1320:40vlaaadWhat I'd really like to have is a tools-deps action, with .m2 and .gitlibs caching etc. Configurable by aliases and deps.edn in a project.#2019-09-1320:32gerredthis is great though!#2019-09-1719:29avi👋 are there any known gotchyas with using gitlibs via t.d.a in CircleCI? I just switched a lib from Maven coords to git coords; it worked fine locally for me but over in CircleCI the deps were downloaded but then the files couldn’t be found on the classpath during the test run.#2019-09-1719:29aviFailed job is here if it’s helpful to see it: https://app.circleci.com/jobs/github/FundingCircle/fc4-framework/2711#2019-09-1719:37ghadi@aviflax hard to say without seeing what bin/tests-with-coverage is doing#2019-09-1719:37ghadiin general if you have private ssh repos, you'll need to use an ssh-agent#2019-09-1719:37aviclojure -J-Xmx2g -A:test:test/coverage
#2019-09-1719:37aviIt’s not a private repo#2019-09-1719:38aviI was able to reproduce the problem just now locally via Docker so I should 😅 be able to figure it out#2019-09-1719:46aviIt looks like the problem is that when the classpath is generated, there’s an extra segment main at the end; that dir doesn’t exist in the repo hosting this particular library#2019-09-1719:46aviHere’s the cp entry: /home/circleci/.gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/58a396f54f469e98b5aae92b9357d65434914228/src/main#2019-09-1719:48aviAnyone have any suggestions on why that might be the case? I’ve been looking through https://clojure.org/reference/deps_and_cli and so far haven’t found any clues#2019-09-1719:48ghadiSeems like that lib has an invalid paths inside its deps.edn?#2019-09-1719:48aviah, maybe, I’ll take a look#2019-09-1719:49aviNope#2019-09-1719:49avihttps://github.com/tatut/clj-chrome-devtools/blob/master/deps.edn#2019-09-1719:53ghadiIt needs to specify paths explicitly#2019-09-1719:54aviOhh wait I remember this#2019-09-1719:54ghadi:paths ["src"]. I think there is a bug when it isn't set#2019-09-1719:54avithe dep is inheriting the source path that was specified in my project root#2019-09-1719:54ghadiYup#2019-09-1719:54aviI mean, my project’s deps.edn#2019-09-1719:54aviIIRC this is a known bug#2019-09-1719:55aviI’ll see if I can find the JIRA issue#2019-09-1719:55dominicmMake sure you vote!#2019-09-1719:56aviwill do, if/when I find it#2019-09-1719:56aviso far no dice#2019-09-1719:56vlaaadwasn't it fixed?#2019-09-1719:57aviaha! yes, it was fixed: https://clojure.atlassian.net/browse/TDEPS-52#2019-09-1719:58aviIt must be that the Docker image I’m using hasn’t picked up the fix yet#2019-09-1719:58aviI’m using circleci/clojure:openjdk-11-tools-deps-browsers#2019-09-1719:58avi@ghadi you reported that bug!#2019-09-1719:59ghadiIt's the circle of life #2019-09-1720:01avi😂#2019-09-1720:01aviThanks for the help!#2019-09-1720:07aviAlex commented on the issue:
> Released in clj 1.10.1.458
And I’ve confirmed that the CircleCI image I’m using has 1.10.0.442 installed.
Interestingly, even the official Docker images for Clojure don’t appear to have any variant with a version of Clojure more recent than 1.10.0.442. This is surprising…#2019-09-1720:10alexmillerthere are no official Docker images for Clojure#2019-09-1720:11gerredi don't even think there should be official Docker images for Clojure in a world of uberjars, to get edgy#2019-09-1720:11aviah, well that explains it#2019-09-1720:11aviwho maintains the images here? https://hub.docker.com/_/clojure#2019-09-1720:11gerredthat's Docker owned, like the company.#2019-09-1720:12aviSo if I think they should be updated, I guess I pester Docker, the company 😬#2019-09-1720:12aviOK thanks all!#2019-09-1720:12gerredyup! if you're interested in custom images and interested in sponsoring some sort of support model, I'd love to chat though.#2019-09-1720:13gerredwe made a #cloud-native channel lately.#2019-09-1720:13gerredso I'm in a process of taking in problems/opinions and trying to distill something#2019-09-1721:11aviNext question:
• I’ve been using Cambada to AOT my code and build an uberjar containing my code and all my dependencies.
• Now that I’ve switched one of my dependencies to a gitlib, Cambada is having trouble building the uberjar.
So: Does anyone happen to know whether there’s an existing solution to this somewhere?
(I mean either some way to continue using Cambada, maybe a fork, etc; or some alternative to Cambada (whether a single tool or a combination of tools) that I might want to consider switching to.)#2019-09-1813:20mike_ananev@aviflax try this fork of Cambada. There are several issues have been fixed. may be your problem is already fixed too.
https://github.com/mikeananev/cambada#2019-09-1813:23aviWill do — thanks!#2019-09-1815:58avi@U097654L8 it doesn’t seem to work; it seems to fail with the same error as the root cambada.
I see you have GitHub’s issues feature disabled in your fork but the feature is enabled in the root repo — would you be interested in a bug report opened as an issue in that repo, which we could use to discuss and debug the problem, and to potentially craft a fix to be merged into your fork?#2019-09-1816:24avi@U097654L8 never mind, I spoke too soon — I took a closer look at the error output and it was not the same error — it was a different error that I’ve seen before and already have a way to work around. And once I applied that workaround, the uberjar process completed successfully. So your fork does in fact resolve my issue with gitlibs. This is great — thank you!#2019-09-1818:33aviFor anyone who might have a similar question: @U097654L8 suggested I try this fork of Cambada: https://github.com/mikeananev/cambada — and it does indeed seem to have fixed the problem I was encountering with gitlibs, AOT, and uberjarring. Thanks Mike!#2019-09-1721:43seancorfield@aviflax Is AOT a requirement?#2019-09-1721:44seancorfieldHere's all the packaging tools available: https://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging#2019-09-1721:44aviI guess it’s not a 100% hard requirement but I’d prefer to preserve it. The project is a CLI tool so it needs to start quickly.#2019-09-1721:45aviThanks, I was just reviewing that list. I was just wondering if anyone had an answer off the top of their head. The descriptions of those tools don’t always include lots of details.#2019-09-1721:45seancorfieldI don't know how many of those do AOT although, in theory, you could always do that step separately and then package stuff: -e "(compile 'my.main.ns)"#2019-09-1721:46seancorfieldI am fairly certain that depstar works with git deps and local deps but it does not do AOT.#2019-09-1721:46aviI believe Cambada is AOT compiling all the Clojure code in my project, including in dependencies. I like that.#2019-09-1721:47aviWould calling compile do the same? I’d guess not...#2019-09-1721:47seancorfieldSince AOT is transitive, I suspect just compiling the main ns will get you much the same result...?#2019-09-1721:47aviIt’s worth trying out for sure!#2019-09-1721:47aviGotta get on my bike now, will check back in shortly. Thanks!#2019-09-1721:48seancorfieldYou'll need to ensure classes exists as a top-level folder in your repo and add it to the :paths vector so it's on the classpath.#2019-09-1800:25cflemingHas anyone made a spec for deps files that I could use to check validity?#2019-09-1800:26cflemingIt’s probably not hard to put together, but I wasn’t sure if anything already existed.#2019-09-1800:26alexmillerI think there are some in tools.deps#2019-09-1800:26alexmillerbut I don't actually use them atm, so they might not be totally up to date (but I'd be happy to fix that)#2019-09-1800:26cflemingOk, I’ll check them out, thanks.#2019-09-1800:26alexmillerhttps://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/specs.clj#2019-09-1800:27cflemingI’d like to make at least a basic check that the file looks good before automatically trying to resolve it on file modification.#2019-09-1800:27alexmiller::deps-map in particular#2019-09-1800:28cflemingYou think ::deps-map might not be up to date?#2019-09-1800:28alexmillerjust eyeballing it, it can't be too far off#2019-09-1800:29cflemingI’m mostly interested validating the input, i.e. the deps file itself, not what t.d.a returns.#2019-09-1800:29alexmilleryeah that ::deps-map is a deps.edn file contents#2019-09-1800:29cflemingAh, ok.#2019-09-1800:31alexmillerI'm pretty sure I'm not using those specs in the mainline code right now so it's probably as good as any code you aren't using#2019-09-1800:31cflemingOk. I might have a go at converting this to my internal validator, which means I could show errors in the editor too.#2019-09-1800:32cflemingAs well as checking before trying to resolve automatically.#2019-09-1800:33alexmillerI don't see anything wrong in those specs. it's missing :sha and maybe one or two other things#2019-09-1800:33cflemingOk, if I see any obvious errors I’ll make a PR#2019-09-1800:33cflemingOr a patch, rather.#2019-09-1800:34alexmilleror just tell me and I'll fix it :)#2019-09-1800:34cflemingThese days, when supplying a patch, does it have to be attached to a JIRA? Or can it be attached to http://ask.clojure.org?#2019-09-1800:34cflemingOr that, yes 🙂#2019-09-1800:35alexmilleras someone that's supposed to be writing a talk right now, I'm in the market for sources of procrastination :)#2019-09-1800:35cflemingMeh, you still have two weeks, what’s the issue?#2019-09-1801:11alexmillerone week!!#2019-09-1802:08cflemingWhat are plane flights for?#2019-09-1800:39cflemingThis is probably more of a question for #spec, but I’m curious about this:
(s/def :local/coord (s/keys :req-un [:local/root] :opt-un [::path]))
#2019-09-1800:39cflemingDoes that mean that :req-un can be used for namespaced keywords as long as the ns is explicitly specified?#2019-09-1800:41cflemingAh, no, ignore me - never mind.#2019-09-1800:42cflemingActually, no - the doc says:
> These variants specify namespaced keys used to find their specification, but the map only checks for the unqualified version of the keys.#2019-09-1800:42cflemingDoes that mean that the spec would accept :root as well as :local/root?#2019-09-1801:02alexmillerha, I was off checking many things and independently discovered this - it's a bug#2019-09-1801:02alexmillerI ran it on all the deps.edn in my system and that barfed at me in a few places#2019-09-1801:09alexmillerfixed and pushed#2019-09-1801:09alexmilleralso added the required :sha attr for git coords#2019-09-1802:00alexmillerreleased as tda 0.7.549 if you need it#2019-09-1802:08cflemingGreat, thank you!#2019-09-1803:17Ahmed HassanWhat is :resource-paths (leiningen) counterpart in Tools Deps?#2019-09-1803:17Ahmed HassanIs it :paths?#2019-09-1803:48cfleming@ahmed1hsn Yes - Deps doesn’t make a distinction.#2019-09-1812:47alexmillerto be more accurate, the jvm doesn't make a distinction, and thus neither does deps#2019-09-1813:19dominicmI stopped making the distinction in my programs too. I realised it was rather arbitrary.#2019-09-1815:41seancorfield@ahmed1hsn You may find this blog post helpful https://corfield.org/blog/2018/04/18/all-the-paths/ since it talks about the terminology used by lein (and boot) and how that maps onto t.d.a.'s world.#2019-09-2022:05shaun-mahoodFor clj-on-windows, how would I start a socket repl? I'm trying powershell -command clj '-J"-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}""' but not sure if I have the quotes wrong or if there is something else going on. The REPL starts up, but I can't get to the socket repl at all.#2019-09-2022:08seancorfield@shaun-mahood Did you try with comma instead of space in that?#2019-09-2022:09shaun-mahoodOf course not - I never think of the Corfield comma 🙂#2019-09-2022:09seancorfieldAlternatively, just use an alias in your deps.edn (which is how I do it on Powershell).#2019-09-2022:10seancorfield(and it has to have commas in the deps.edn version because of the round-tripping through the .cpcache files and shell etc)#2019-09-2022:15seancorfield@shaun-mahood Did it work with the comma?#2019-09-2022:16shaun-mahoodNo luck yet - I might be putting the commas in the wrong place though#2019-09-2022:18shaun-mahoodI just realized I have to put lein on that machine, though, so I might use that instead - saves me from figuring out how to get Cursive to connect to a socket repl#2019-09-2022:19seancorfieldCursive requires nREPL doesn't it? So you wouldn't want to start a Socket REPL anyway.#2019-09-2022:20shaun-mahoodYeah, I think there's a way around that (using https://github.com/mfikes/tubular), but nREPL is the way to go for this one I think.#2019-09-2022:21seancorfieldclj -A:nrepl 🙂#2019-09-2022:21seancorfield:nrepl {:extra-deps {nrepl {:mvn/version "RELEASE"}}
:main-opts ["-m" "nrepl.cmdline"]}
#2019-09-2022:22shaun-mahoodYeah that looks nice and easy 🙂#2019-09-2022:22seancorfieldhttps://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L47-L57#2019-09-2022:23seancorfieldThis is why I have so much stuff in my .clojure/deps.edn file -- and I have it on every machine and under every shell (macOS, Windows Powershell, WSL, etc)#2019-09-2022:54shaun-mahood@seancorfield It looks like it was a firewall issue, not a Clojure issue - thanks for the help, using your aliases I at least found where the problem was. Now to figure out why Windows Server is ignoring me when I tell it to open a port...#2019-09-2514:38val_waeselynckIs it possible to deps-consume a JAR that's versioned in a Git repo?
Context: I have a little private java library that I want to consume in a deps.edn project. Putting the generated JAR under version control is weird, but this seems to be a minor inconvenience compared to setting up a private Maven distribution infrastructure.#2019-09-2514:50dominicmIf something was going to work, it would be :deps/root#2019-09-2514:54val_waeselynckLike this?
{:git/url "#2019-09-2514:54val_waeselynck(Assuming the JAR was generated at the root of the project).#2019-09-2520:09alexmillerThat won’t work#2019-09-2605:36val_waeselynckThanks @U064X3EF3. Do you see any other way?#2019-09-2607:27alexmilleryou could add the jar file to your :paths in the deps.edn of the git dep#2019-09-2611:04dharriganHello! Could master of tools.deps.alpha be merged into add-libs please? The add-libs is of tools.deps.alpha-0.7.541 (August 9th), not tools.deps.alpha-0.7.549 (September 17th)?#2019-09-2611:04dharriganThank you very much 🙂 I guess, any reason why add-lib can't be brought in to master? It's super useful 🙂#2019-09-2611:11alexmillerI don’t think you’re missing anything from master - just some tweaks in the specs (which aren’t used)#2019-09-2611:12dharriganfantastic! 🙂 Thank you (for clarifying!)#2019-09-2611:13alexmillerThere are issues with add-lib in some environments and it’s unlikely to end up included in its current form (some of it may actually be moved into Clojure itself)#2019-09-2611:14dharrigankk, that's good to know. Havin the ability to dynamically pull in a dep - for experimentation on the repl, is amazingly useful to have 🙂#2019-09-2611:14dharrigan(and given clojure's great interop, doing that playing around is a very happy process!)#2019-09-2612:14dominicmWhy move it into clojure @alexmiller? Which bits?#2019-09-2613:01alexmillerthe part that tracks the current “libs” currently in use could be built in. It’s already designed to be initialized in a couple ways so even non tools.deps tools could convey the lib set via those means. add-lib itself relies on tools.deps so that probably cannot move, but the parts that update the lib set could be genetically useful#2019-09-2613:02alexmillerGenerically :)#2019-09-2613:06dominicmWhat's the goal of making the concept of a library first class? I can imagine using it to affect requires in different ways (e.g. Namespace versioning). Is there a set of goals in mind yet?#2019-09-2613:08alexmillerNo, none of this is baked which is why it’s on a branch#2019-09-2613:56alexmillerThere are a lot of potentially interesting things that could be done though#2019-09-2715:27plexusseems Clojure CLI uses a jar that contains Clojure + tools.deps. How would I build such a jar myself? or put in a different way, how can I make clojure use my local changes to tools.deps to try them out?#2019-09-2715:39alexmillerThe easiest way (what I do) is to just hack my clojure script to put my local source directory at the front of the tools classpath#2019-09-2715:39alexmillerThen you can pick up changes with no build at all#2019-09-3004:08taylorI have a deps.edn project/repo that I’d like to also keep example projects, but ideally those wouldn’t get pulled when another deps.edn project depends on mine e.g. via git. Would tools.deps pull those “extra” files from my repo, and the README and everything? Should I stop worrying about a few kilobytes?#2019-09-3004:14seancorfield@taylor If someone depends on your project via git deps, the whole repo will be checked out (regardless of which parts are actually used).#2019-09-3004:15seancorfieldIf you publish an artifact (on Clojars or wherever) then you can choose what goes into that.#2019-09-3006:43sogaiuyou get a bare clone of the repository + one or more checkouts corresponding to whatever commits are used, right? at least, that's how it looks under ~/.gitlibs/#2019-09-3012:49alexmillerYes, that’s correct#2019-10-0100:36plexusIf there are gitlab users here that would be interested in using the private Maven repositories provided by gitlab, then please check out this patch and provide feedback. https://clojure.atlassian.net/browse/TDEPS-140#2019-10-0100:38plexusGitlab uses a token in a http header for authentication instead of the normal username/password method, this patch adds support for that.#2019-10-0104:17mattyulrichis it possible to resolve a pom type dep (rather than a jar) in my deps.edn file?#2019-10-0104:21mattyulrichI think in lein, one could specify :type "pom" and it would resolve the pom alone without requiring a jar at the end...#2019-10-0104:29mattyulrichoh.. nevermind - I just found the :extension "pom" attribute - did the trick perfectly!#2019-10-0112:17kirill.salykinHi
I am playing with Github actions and tools.deps, seems there is an issue with installing deps from private github repositories
Did anyone also experienced it? Do I need to install my ssh key?#2019-10-0112:23alexmillerYes, you would need to do that#2019-10-0112:23kirill.salykink, thanks#2019-10-0112:23alexmillerI believe there is a special env var for it automatically in the action#2019-10-0112:24alexmillerBut you’d need to add it to the ssh-agent#2019-10-0112:24alexmillerI haven’t done this yet myself so don’t have a recipe for you #2019-10-0112:24kirill.salykinthanks, will try to install ssh key and add it to agent#2019-10-0112:59kirill.salykinadding ssh key didnt help much
name: CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/#2019-10-0112:59kirill.salykinError building classpath. connector is not available:
com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available:
at com.jcraft.jsch.agentproxy.ConnectorFactory.createConnector(ConnectorFactory.java:120)
#2019-10-0112:59kirill.salykinplease advice what can be wrong?#2019-10-0114:34ghadi@kirill.salykin in Github Actions, do processes persist between steps?#2019-10-0114:42kirill.salykinI don’t know, to be honest
testing now#2019-10-0114:55kirill.salykinyes, you are correct#2019-10-0114:55kirill.salykinrun: eval $(ssh-agent -s) && ssh-add -k ~/.ssh/id_rsa && ./bin/run-tests-ci.sh
this works#2019-10-0115:02kirill.salykin> My guess is eval $(ssh-agent -s) dies at the end of that step?
this is my guess as well#2019-10-0115:03kirill.salykinhttps://medium.com/tokenmill/how-to-build-a-clojure-project-that-depends-on-private-git-repositories-in-docker-635c7a576726#2019-10-0114:35ghadiYou can find out by running
- run: ssh-add -l
in a subsequent step#2019-10-0114:35ghadiI have github actions building private git deps, but I'm doing it all within a container#2019-10-0114:39kirill.salykini am able to clone that repo via just git#2019-10-0114:40ghadibut tools.deps isn't using git#2019-10-0114:40kirill.salykinyeah, just to ensure that I can clone it#2019-10-0115:02ghadiMy guess is eval $(ssh-agent -s) dies at the end of that step?
@kirill.salykin#2019-10-0115:02ghadioh just saw the thread#2019-10-0115:04ghadiyou should probably pin the https://github.com host key statically in source code#2019-10-0115:04ghadiotherwise it will accept any known host key -- MITM attack#2019-10-0115:04kirill.salykintrue, thanks#2019-10-0316:58dominicm@alexmiller bit of a weird one https://github.com/juxt/pack.alpha/issues/64 no repro yet exactly, but it looks like RepositorySystem is coming back as nil. I don't suppose you're clued up on this and can think of an obvious cause?#2019-10-0317:28alexmillerI looked at that when it was filed as it was mentioned here too. It's weird, don't know what to make of it. I think from the stack this pre-dates the session cache additions?#2019-10-0317:30dominicmIt's still happening after updating too.#2019-10-0317:30dominicmDigging in, the impl of the service locator does attempt to add the repository system, so that should be there.#2019-10-0317:31dominicmBumping from 1.1.1 of the maven dependencies to 1.4.1 doesn't fix it either.#2019-10-0317:37dominicmthis is the problem with service locators. All the damn coupling.#2019-10-0317:45dominicmhttps://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/tools-deps/near/159950271#2019-10-0317:57Joe LaneHey friends, I'm having an issue cloning my own private github repo with tools deps. Even though I'm able to clone the repo with the ssh url into a local tmp directory, JGit is still telling me that Caused by: org.eclipse.jgit.errors.TransportException: .
I'm on a mac with these version numbers:
Joes-MacBook-Pro:snapestry-ion magemasher$ clj -Sverbose
version = 1.10.1.469
install_dir = /usr/local/Cellar/clojure/1.10.1.469
config_dir = /Users/magemasher/.clojure
config_paths = /usr/local/Cellar/clojure/1.10.1.469/deps.edn /Users/magemasher/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/479583893.cp
The ssh key I have on github matches this key below:
ssh-add -l -E md5
4096 MD5:10:ef:06:6b:9b:94:86:fc:06:83:9f:17:0a:8c:bb:10 /Users/magemasher/.ssh/id_rsa_jplane (RSA)
Does anybody have any ideas? I'm really stumped here.#2019-10-0317:57Joe Lane#2019-10-0317:58alexmilleris ssh-agent running?#2019-10-0317:59Joe Laneyes#2019-10-0317:59Joe Lane(I just confirmed by starting it, same failure)#2019-10-0318:00alexmillerthere are some known failure modes with host key algorithms etc, but this doesn't really look like that#2019-10-0318:00alexmillerI assume you can git clone with that key#2019-10-0318:00Joe Laneyep#2019-10-0318:01ghadiDo you have entries in .ssh/config ?#2019-10-0318:01ghadiSometime jsch tries to read those instead of the ssh agent#2019-10-0318:01alexmillersome tips at https://clojure.atlassian.net/browse/TDEPS-91#2019-10-0318:02Joe LaneI do, are they safe to paste?#2019-10-0318:02ghadiYes#2019-10-0318:02ghadiIt’s usually just file names#2019-10-0318:02alexmillerthere's one like IdentityKey or something that might need to be removed#2019-10-0318:02Joe LaneHost *
AddKeysToAgent yes
IdentityFile ~/.ssh/id_rsa_jplane
IdentitiesOnly yes
Host
HostName
User admin
IdentityFile ~/.ssh/id_rsa_jplane
#2019-10-0318:03alexmilleryeah, remove the IdentityFile line in first block#2019-10-0318:03ghadiComment out the first block#2019-10-0318:03Joe LaneHey, it works!#2019-10-0318:04Joe Lane(commenting out just the IdentityFile) So, the reason I had that line in there in the first place was because of these instructions. https://help.github.com/en/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#2019-10-0318:05Joe LaneSo, while I'm glad this was fixed for me, are the github instructions conflicting with what Jsch needs?#2019-10-0318:08Joe LaneEither way, thank you both for your assistance! I see similar advice was given by Alex on the jira issue.#2019-10-0318:09dominicm@alexmiller I managed to wrangle a stacktrace https://github.com/juxt/pack.alpha/issues/64#issuecomment-538061024#2019-10-0318:10dominicmThe problem is, I can't figure out what would set a version resolver! There's nothing in the default service locator code.#2019-10-0318:11ghadi@lanejo01 JSch is going to be a constant source of headaches#2019-10-0318:38dominicmOkay. Figured out what sets the version resolver, but I don't understand how I'm getting java.lang.ClassCastException: Cannot cast org.apache.maven.repository.internal.DefaultVersionResolver to org.eclipse.aether.impl.VersionResolver when defaultversionresolver is like this:
public class DefaultVersionResolver
implements VersionResolver, Service
#2019-10-0318:42dominicmDoesn't that explicitly state that it's a VersionResolver?!#2019-10-0318:44seancorfieldIs it crossing a class loader boundary perhaps?#2019-10-0318:56alexmilleryeah, must be same class loaded from different loader#2019-10-0318:57alexmillerlooks like you're buried in some capsule stuff - doesn't that do weird things with classloaders?#2019-10-0318:59alexmilleror is this just setting up capsule stuff#2019-10-0319:04alexmillerI'm not sure what the setup is, but there must be same class available via different loaders somehow here#2019-10-0319:05dominicmThis is setting up capsule stuff, so no classloaders involved here yet.#2019-10-0319:05dominicmcapsule is very good on the classloader front fwiw btw. It just extracts to a directory and runs it, much like leiningen. OneJar is the classloader one.#2019-10-0319:10dominicmI'll see if I can make this a minimal repro at least. Remove pack from the equation.#2019-10-0319:16dominicmrlwrap: Oops, crashed (caught SIGABRT) - this should not have happened!If you need a core dump, re-configure with --enable-debug and rebuildResetting terminal and cleaning up...
Today is not my day apparently! 🙂#2019-10-0319:22alexmillerwow#2019-10-0319:22ghadithat happens a lot with long lines#2019-10-0319:23ghadiprobably a method to constrain it. haven't looked#2019-10-0319:23ghadilong like 10MB lines, in my experience#2019-10-0319:29andy.fingerhutI sometimes idly wonder how many different behaviors exist among line-based utility programs for very very long lines.#2019-10-0320:51dominicmWhat I don't understand is how simply commenting out oz would change things. How does that cause a classloader?!#2019-10-0320:52alexmillerI have no idea#2019-10-0320:59alexmillerI don't have a theory for why this is happening, just trying to work backward from the evidence#2019-10-0321:00dominicmI can't create a minimal repro either. Taking the exact pack code and placing it in another project stops it from happening... 🤔#2019-10-0321:00alexmillercan you get pack out of it? Just call tools.deps with oz stuff?#2019-10-0321:00alexmillerI can't remember if I did that when it was logged or not#2019-10-0321:01dominicmDoing that doesn't throw. There's something contextual/environmental that I am not accounting for.#2019-10-0321:02alexmillerare you running via clj? if so, what's the classpath? Are there any uberjars such that same class has two possible sources?#2019-10-0321:04dominicmRunning via clj. The classpath is pretty big because of oz.#2019-10-0321:22dominicmduplicates looking likely#2019-10-0321:23dominicmclojure -Spath -A:pack:packit | (while IFS= read -rd: line; do jar tf "${line}"; done) | sort | uniq -d I did this and got results#2019-10-0321:38dominicmI know why my repro isn't working, it's because I'm not co-locating oz and t.d.a on the classpath in my repro, but I am when I load pack into an existing project. That's also why this happens when used with add-lib.#2019-10-0321:40dominicmIt's nothing to do with dependency resolution (not really anyway). It's to do with the classpath tda is running under.
I saw maven duplicates with the above classpath, so I'm guessing that's the root cause. Although I've never seen issues like this where one wins out over the other in different places.#2019-10-0322:07hiredmanthe classpath isn't canonical, classloaders are, the classpath is interpreted by some classloader, so if some wants a classloader that behaves sort of like the system classloader (or whatever the root classloader is called these days) it can make a classloader that interprets the classpath but has a slightly different behavior#2019-10-0322:21alexmillerMaven has all kinds of magic dependency injection and it’s entirely possible there is some service config somewhere that is giving weird behavior. Usually I just avoid using all of that stuff when I call the maven APIs but it’s surprisingly invasive.#2019-10-0407:35dominicmThe alternative in this case is creating impl classes, and I'm not sure if that's better or worse. It's unclear whether that's a stable api to me, because the artefact is named impl too#2019-10-0407:57dominicmSo the repro is this:
;; deps.edn
{:paths ["src"]
:deps
{org.clojure/tools.deps.alpha {:mvn/version "0.7.541"}
metasoarous/oz {:mvn/version "1.6.0-alpha5"}}}
;; src/io/dominic/tda_npe/core.clj
(ns io.dominic.tda-npe.core
(:require
[clojure.tools.deps.alpha :as tools.deps]))
(def deps-map
'{:paths ["src"]
:deps {org.clojure/clojure #:mvn{:version "1.9.0"} }
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}})
(defn -main
[& _]
(tools.deps/resolve-deps deps-map {}))
#2019-10-0408:23dominicmclojure -Spath | while IFS= read -rd: line; do
if [ -f "${line}" ]; then
jar tf "${line}" | while read -r file; do echo "${line}: ${file}"; done
fi
done | sort -k2 | uniq -f1 --all-repeated=separate
Here's a script for finding duplicates on the classpath 🙂#2019-10-0408:33dominicmmetasoarous/oz {:mvn/version "1.6.0-alpha5"
:exclusions [org.apache.maven/maven-aether-provider]}
This fixes it (haven't tested oz). Looks like there's 2 sources for the default impl of VersionResolver!#2019-10-0408:34dominicmIt's coming transitively from pomegranate, so I think you can't have pomegranate and t.d.a on the classpath together without this exception.#2019-10-0411:06delaguardohttps://github.com/marketplace/actions/setup-clojure
I wrote github action to setup hosted runner with clojure cli.#2019-10-0412:02vlaaadThis is amazing, thank you very much!#2019-10-0412:42delaguardoyou are welcome to open an issue if something isn’t working as expected)#2019-10-0412:49vlaaadIntegrated it into cljfx, works great! https://github.com/cljfx/cljfx/blob/master/.github/workflows/test.yml#2019-10-0416:52delaguardoalso published action for provision graalvm instead of java - https://github.com/marketplace/actions/setup-graalvm-environment#2019-10-0418:34seancorfieldAdding this to next.jdbc to matrix test on Java 8 and Java 11. Thank you!#2019-10-0418:52seancorfieldWow, it takes a long time for those jobs to get started! 21 minutes for the first one. Still waiting for the second one in the matrix.#2019-10-0419:04seancorfield29 minutes end-to-end. It'll be interesting to see how they tune that during the beta. CircleCI started almost immediately (and completed in 30 seconds, compared to an actual run time of about a minute on GitHub).#2019-10-0419:36vlaaadFor me it usually starts immediately...#2019-10-0419:37seancorfieldMaybe it was just because this was the first time I've used it? I'll report back on subsequent runs.#2019-10-0420:54seancorfieldJust pushed another commit and, yeah, it started up nearly immediately.#2019-10-0510:56ikitommihi. I have a transitive deps issue. metosin/malli depends on both borkdude/sci and borkdude/edamame via sha. sci uses edamame via real mvn version. When I try to use malli, the versions conflict:
➜ ~ clj -Sdeps '{:deps {metosin/malli {:git/url "" :sha "28b4ee63bd571ce612e8cd0e9dab6ef86722597e"}}}'
Checking out: at 28b4ee63bd571ce612e8cd0e9dab6ef86722597e
Downloading: borkdude/edamame/0.0.6/edamame-0.0.6.pom from
Error building classpath. Unable to compare versions for borkdude/edamame: {:mvn/version "0.0.6", :deps/manifest :mvn} and {:git/url "", :sha "9b3e825b65ec6fe0dc536988fd036e6e681813de", :deps/manifest :deps, :deps/root "/Users/tommi/.gitlibs/libs/borkdude/edamame/9b3e825b65ec6fe0dc536988fd036e6e681813de"}
#2019-10-0510:58ikitommiIf I add explicit sha to edamame from command like, it works, but is not what I would like to be needed.#2019-10-0511:00ikitommie.g. this works:
➜ ~ clj -Sdeps '{:deps {metosin/malli {:git/url "" :sha "28b4ee63bd571ce612e8cd0e9dab6ef86722597e"}, borkdude/edamame {:git/url "", :sha "9b3e825b65ec6fe0dc536988fd036e6e681813de"}}}'
#2019-10-0511:21ikitommiquick-hacked this so that malli depends on edamame mvn version. Still, curious could it be made to work with just Sha dep to it#2019-10-0511:27dominicmWhat do you think should happen here? The problem is that maven and git don't have a comparison mechanism that makes sense.#2019-10-0511:34borkdude@ikitommi I think it's best to depend on a mvn version of edamame then from now on. I'll try to do that with sci as well.#2019-10-0512:01alexmillerThis can be made to work, but I haven’t done so yet. Maven artifacts generally record their sha at build time so it is possible to version compare them.#2019-10-0512:02borkdudewhat if you have two branches that are diverged from the same maven version?#2019-10-0513:58alexmillerThis is just back to the same rules we use for git comparisons - if one version is a descendant of another, its preferred. If there is no ancestor relationship, they aren’t comparable#2019-10-0513:59alexmillerIn that case, it will throw and you need to be explicit at the top level about which dep to use (top dep version always used)#2019-10-0715:52alexmillerif anyone here is an add-lib user, can I ask the scenario where you're using it? is it
1) to definitely add a lib you know you need from prior experience
2) to add a lib you might want to use but are not sure if you need
3) to do a task at the repl that you definitely don't want in your deps.edn#2019-10-0717:16seancorfieldIs 3) meant to include situations where you have a "scratch" REPL running just to try stuff out (e.g., helping beginners on Slack) and it would be less convenient to restart that REPL with difference deps rather than just adding a lib to the REPL? For me, such situations are usually not accompanied by deps.edn at all.#2019-10-0717:18alexmilleryou always implicitly have a deps.edn, even if it's just including clojure from the system deps.edn#2019-10-0717:19alexmillersounds like a mixture of 2 and 3#2019-10-0717:19seancorfieldWell, yeah, I meant the "project" (current directory) one. Libraries that I use add-lib for wouldn't generally go in either of the other files.#2019-10-0717:20seancorfieldI have a scratch directory that has deps.edn with {} in it and I usually start a plain REPL with -Sdeps on the command-line, but if add-lib was easier to use, I'd just start that plain REPL and then add-lib in anything I wanted to experiment with and just leave it running (instead of restarting with different deps).#2019-10-0807:47plexusFor me it would be mostly 1), especially in the early stages of a project I want to be able to add dependencies without having to restart the process. It could also be 2) if it's a lib I haven't used before, to be able to quickly evaluate it before committing to it.#2019-10-0715:56dominicmI have played around with the idea of 3, maybe even for use in dev tooling libraries. But I haven't actually done it yet.#2019-10-0715:58alexmillerby which you mean you would have a lib that called add-lib?#2019-10-0716:00dominicmYeah. Although essentially it's a library of things you'd do at the repl.
I can't remember the details of why I was doing this now. Possibly to keep things "light" at someone's request, or maybe because I wanted to load things for dev time based on the config.edn#2019-10-0715:57alexmillerand if you answered 1 or 2, how do ensure the dep gets added to deps.edn? do you do it at the same time you add-lib or rely on remembering later?#2019-10-0715:58delaguardotests#2019-10-0715:59alexmillernot sure what that means? you add tests and rely on that failing later to tell you to add the lib?#2019-10-0716:01delaguardoyes, not a perfect solution, but covers most of the problems
something like Could not locate chesire/core__init.class, chesire/core.clj or chesire/core.cljc on classpath. usually is enough to remember what went missing#2019-10-0716:11kennyYep, pretty much the same thing as @U04V4KLKC. If the lib ends up getting used in the project, it will get added to a namespace's :requires. This will cause the project to fail CI tests if someone forgot to add it to the deps.edn.#2019-10-0716:40dharriganremember later#2019-10-0717:14seancorfieldFor 1) I do it at the same time because it's a deliberate act. For 2) I do it later, once I've decided whether the lib is really what I need (so, yeah, I run the risk of forgetting but if I really need the lib, it'll end up required in the code base somewhere so tests will fail if I have forgotten to add it).#2019-10-0715:58dominicmLater. Although I've been working on something which would remember what is add-lib'd and allow me to commit it. And also an alternative entry point for loading it and committing simultaneously.#2019-10-0715:59alexmillerI'm currently exploring similar ideas#2019-10-0716:01dominicmAs an fyi, I currently don't find the git tag rewriting useful.
I put comments into my deps.edn, and losing those is annoying. I've got a few toys that use rewrite-clj to avoid that (including a implementation of the git tag stuff), but they're difficult to write.#2019-10-0716:03dominicmOr rather, it's useful, but that side effect makes it problematic.#2019-10-0716:03alexmilleryeah, I'm probably going to remove that#2019-10-0716:03dominicmI know :) but the external version would have the same problem, so would a commit-add-lib!#2019-10-0716:04alexmillerwell, I'm more exploring a "refresh" type option#2019-10-0716:04alexmillerso you would be the one doing the add#2019-10-0716:04dominicmAh sync-deps, I think someone tried that here. They requested knowledge of the aliases in use so they could merge them together :)#2019-10-0716:08alexmiller"sync" is ambiguous about the direction of the sync. "refresh" is deps.edn -> repl#2019-10-0716:13dominicmThat's fair. Refresh made me think of tools namespace and I struggled to separate the two.#2019-10-0717:16alexmillerHow do you know the version of the lib for add-lib? Do you:
1) Look it up on github or other source
2) Use LATEST or RELEASE#2019-10-0717:17seancorfieldI use "RELEASE" a lot when I'm just experimenting with a library. I only bake the version into deps.edn once I know I'm going to use it.#2019-10-0717:18seancorfieldThe "refresh" scenario wouldn't cover a "scratch" situation where you're just trying a lib out in the REPL and don't want deps.edn created/updated.#2019-10-0717:23alexmilleryes, I'm aware. investigating alternatives, making tables :)#2019-10-0717:20dharriganMostly 1, sometimes 2.#2019-10-0717:28dominicmI usually look it up, I need to look up the coord anyway. I don't memorize those.#2019-10-0806:12orestisJust a data point - sometimes I’m repl’ed in a remote environment and I would find it useful to add a library without having to touch the remote deps.edn file, which might be tucked in a jar in any case. It’s on me to remember to do it after the fact, and that would require a proper deploy step anyhow. #2019-10-0806:14orestisI’m sure the table Alex is building will be much more nuanced, but the obvious (to me) building block is a function that can add a fully specified dep to the runtime. Figuring out the latest version, refresh the env from a deps.edn could be userland concerns, no?#2019-10-0813:19dominicmThis is what we have, so I think these might be baked in as well#2019-10-0821:49Michaël SalihiHi ! I I would like to know if it's possible to populate deps.edn dependencies in command line or some externals tools ?
Right now, I use clj-refactor with cljr-add-project-dependency in Emacs but it only works when a REPL is already running.#2019-10-0822:23ghadi@admin055 it’s just data - you can programmatically generate it#2019-10-0901:43mpisankohi there, pardon if totally missed this (but google didn't give me any answers). my question is: is there an alternative to lein's pedantic? in tools.deps#2019-10-0902:14seancorfield@mpisanko No. It's also worth bearing in mind that the algorithm for selecting versions is different in t.d.a. so there should be fewer potential conflicts. Or at least, fewer counterintuitive version selections 🙂#2019-10-0902:16seancorfieldI think there's still an undocumented :verbose true option you can add to deps.edn so that t.d.a. explains all of its decisions...#2019-10-0902:17mpisankoright, that's the premise of improved dependency resolution i guess. thanks @seancorfield!#2019-10-0902:42seancorfieldLooks like :verbose needs to be under an alias, not at the top-level... {:aliases {:v {:verbose true}} ,,,} and then clojure -A:v will show the resolution output as t.d.a. works.#2019-10-0902:57alexmilleractually there are tools that can help you with this in clj#2019-10-0902:57alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/Tools is a big list#2019-10-0902:57alexmillerin particular, check out depot or deps-ancient#2019-10-0903:01seancorfieldThose will detect old (outdated) deps in your project -- lein pedantic detects possible version conflicts (usually in transitive deps) and warns about them (and prevents command execution if any potential conflicts are present, as I recall) so it's not really equivalent to anything on that list...?#2019-10-0903:02seancorfieldI think t.d.a.'s approach makes much more sense and you can either eyeball things with -Stree to see what actually got picked, but you can only see the potential conflicts if you ask t.d.a. to be verbose, as far as I can tell.#2019-10-0903:03seancorfield(we use depot at work, and it's very useful for flagging deps that could be updated -- but does not show potential conflicts)#2019-10-0903:10seancorfieldLooking in more detail at some verbose output from t.d.a. I can see its decision process cuts off some paths that I think Leiningen considers... I'd have to try to repro in lein to be sure... (this has piqued my curiosity!).#2019-10-0903:20seancorfieldYes, confirmed. Example: deps.edn specifies Clojure 1.9.0 and seancorfield/next.jdbc 1.0.8. t.d.a. pins those versions because they are top-level and when it sees that next.jdbc depends on Clojure 1.10.1 it just discards that due to 1.9.0 being the top-level dep. Consequently, t.d.a. goes on to select spec.alpha and core.specs.alpha based on Clojure 1.9.0. When you put the same deps in project.clj and :pedantic? :abort and ask for lein deps :tree, Leiningen also shows the transitive dependencies of Clojure 1.10.1 and warns they conflict with Clojure 1.9.0's deps, suggesting exclusions (on next.jdbc) to exclude all three of clojure, spec.alpha, and core.specs.alpha (and then it aborts).#2019-10-0903:21seancorfield(if you tell lein to exclude org.clojure/clojure that is enough to satisfy it)#2019-10-0907:48dottedmag@alexmiller Please have a look at TDEPS-56 when you have time. I have posted a patch and an upsides/downsides table some time ago, and there was no further discussion on the topic.#2019-10-0907:59Michaël Salihi@ghadi Yes, I wanted to know if there was already existing tool to manage dependencies before to begin development.
I find this tool who do the job https://github.com/hagmonk/find-deps#2019-10-0920:30jrwdunhamIs it possible to use an alias to override a github library with a local root library?#2019-10-0920:32jrwdunham{:deps {github-myorg/myrepo {:git/url "some-url" :sha "some-sha"}}
:aliases {:local {:override-deps {myrepo {:local/root "/abs/path/to/my/local/repo/"}}}}}#2019-10-0920:33jrwdunham^ doesn't seem to do what I want.#2019-10-0920:33jrwdunhampresumably because the lib names github-myorg/myrepo and myrepo differ#2019-10-0920:36jrwdunhamd'oh. It looks like I can just stop using the symbol github-myorg/myrepo for the GH dep and instead use myrepo as with the local dep. I didn't realize that was just a naming convention and not a requirement.#2019-10-0920:42dominicmExtra-deps instead of override#2019-10-0920:42jrwdunhambut wouldn't that result in two deps with the same namespaces?#2019-10-0920:43jrwdunhamto be clear, this now seems to do what I want:#2019-10-0920:43jrwdunham{:deps {myrepo {:git/url "some-url" :sha "some-sha"}}
:aliases {:local {:override-deps {myrepo {:local/root "/abs/path/to/my/local/repo/"}}}}}#2019-10-0920:48seancorfieldYup, myrepo in the above is the coordinate for your dependency: the group and artifact ID, as if it were myrepo/myrepo -- and the coordinate has to match since that's what you are overriding.#2019-10-0920:49seancorfield(well, technically, the "coordinate" is the group/artifact and the version information but...)#2019-10-0920:50seancorfieldSo it isn't "just a naming convention" -- it has a specific meaning.#2019-10-0920:57jrwdunhamThanks @seancorfield that clarification is helpful. I should have been more clear: I intended to write that I was under the false assumption that if a coordinate used the :git/url key in its version info that it had to have a group containing the github- prefix.#2019-10-0921:00seancorfieldIf you're not trying to override a "regular" dep with a git dep, then you can give it any unique name you want, but using your github organization (username) and the repo name means it's not likely to conflict with a "regular" dep.#2019-10-0921:03seancorfieldIf you're working with a regular dep (group/artifact) with a Maven version, you can just swap out the version part for a :git/url part or :local/root part and keep the same group/artifact to effectively override the Maven version with that git or local version -- and that will work even if it is a transitive dependency of something else you're depending on (since you'd have it as a top-level dep).#2019-10-0921:03seancorfieldDoes that help, or just make it more confusing @jrwdunham?#2019-10-0921:05seancorfieldAn example: https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L136-L149 -- that uses a git version instead of the Maven Central version of org.clojure/tools.deps.alpha#2019-10-0921:15jrwdunham@seancorfield I am left a bit more helped and a bit more confused. So, in your example, since you are not declaring a top-level :deps for org.clojure/tools.deps.alpha, you use :extra-deps in your :deps alias to effectively override the Maven dependency that would be installed otherwise. Is that accurate?#2019-10-0921:52seancorfieldI use :extra-deps there to ensure it is added (with that git version). :override-deps only takes effect if something else is bringing in that dependency.#2019-10-0921:54seancorfield(in this case, it becomes a top-level dep so even if something else brings in t.d.a. as a transitive dep, my selection takes precedence)#2019-10-0923:24cflemingHas anyone had problems using s3 repos with deps under Java 11?#2019-10-0923:25cflemingSome Cursive users are having problems which is reported as not being able to access java/sql/Timestamp, when using lein that’s usually resolved by upgrading the s3 wagon. But it looks like the deps version is baked in.#2019-10-0923:26cflemingIs anything required to be added to the module classpath, or anything like that?#2019-10-0923:30alexmillerno, lots of people doing this, haven't heard any issues#2019-10-0923:30alexmillerjava/sql/Timestamp is in particular not in the base module#2019-10-0923:30cflemingOk, I’ll see if I can figure it out, and I’ll report back if I do.#2019-10-0923:30cflemingRight, that’s what’s been giving problems since IntelliJ started using JBR11 as its base JRE.#2019-10-0923:31alexmillerso lein's trick of adding itself to the bootclasspath doesn't work#2019-10-0923:31alexmilleras at boot time, only the base module is loaded#2019-10-0923:31cflemingI’m pretty sure it’s not a bootclasspath issue, since this is executed within Cursive and I don’t use it.#2019-10-0923:31cflemingWell, IntelliJ doesn’t use it, more accurately.#2019-10-0923:32alexmilleryeah, didn't make sense to me either#2019-10-0923:32cflemingI run lein in an isolated classloader using shimdandy.#2019-10-0923:32alexmillerbut it matches those symptoms#2019-10-0923:32alexmillerwhat do you put in the isolated classloader? and where is it rooted?#2019-10-0923:32alexmillerif it's rooted off the base loader or app loader or whatever they call it, it might only have the base module available#2019-10-0923:35cflemingYes, I think that’s probably the case. It’s rooted off the parent classloader for a boostrap class I use, which IIRC is the app classloader. It almost certainly won’t have the SQL module available.#2019-10-0923:36alexmillerin java 9+ that loader is represented as null (the bootstrap loader). it only has access to the base module#2019-10-0923:36cflemingBut my impression was that some of the older s3 wagons had problems with this that were resolved in later versions - certainly using lein, upgrading the wagon usually fixes the issue.#2019-10-0923:36alexmillerdeps uses a newer version of the s3 wagon than the lein plugin does#2019-10-0923:36cflemingBut my understanding of how all this works in the brave new world is sadly incomplete. Looks like I’m going to have to figure it out.#2019-10-0923:37cflemingHmm, interesting.#2019-10-0923:37alexmillerwell, maybe it's been updated since I last looked, not sure#2019-10-0923:37cflemingOk, I’ll see if I can reproduce the issue and debug it.#2019-10-1114:25rapskalian#2019-10-1114:28rapskalianIf my lib makes use of a dep, but I fully expect the users of the lib to also make use of that dep, it made sense logically to not include those deps as transitive dependencies. But I lack a full understanding of how t.d. resolves conflicts…#2019-10-1114:28rapskalianI probably just need to RTFM 🙂#2019-10-1114:32alexmillerthis is certainly under-documented#2019-10-1114:33alexmillerIn general, I think it makes sense for a lib to include all the dependent libs it needs in its explicit dependencies#2019-10-1114:34alexmillerthe way tda resolves conflicts is generally to pick the newest version#2019-10-1114:34alexmillerwith the caveat that it picks whole subtrees and in some cases that upstream libs higher in the tree may dominate#2019-10-1114:34alexmillerhowever, one big hammer is that the versions at the very top (presumably the app) always win#2019-10-1114:35alexmillerso the end user always has final say#2019-10-1114:36alexmillerso if the app includes both your lib and the datomic cloud-client lib, then it's explicitly naming the datomic lib and that version will be used, which is what you want here#2019-10-1114:36alexmillerlet me know if that doesn't make sense#2019-10-1114:37rapskalianOkay, that makes sense, thanks. Explicit deps are certainly much simpler as well. It’s nice to be able to lean on t.d.a. a bit there.#2019-10-1114:38rapskalianI’m going to remove my :provided alias for now and just go with the simplest solution. Thanks, Alex.#2019-10-1114:57dominicmThe only thing close to provided deps would be optional specs.#2019-10-1117:01rapskalian@dominicm is there a t.d. way to do that? Best I’ve seen people do is add a conditional flag surrounding calls to s/def…#2019-10-1117:02dominicmNo, nothing special. Just that or optional namespaces.#2019-10-1117:36alexmillert.d way to do what?#2019-10-1120:37rapskalian@alexmiller to optionally also bring in the specs…I’m not sure exactly what that would mean honestly. Potentially some way to specify a dependency along with additional aliases, like:
:deps {me/sweet-dep {:mvn/version "XYZ" :aliases [:specs]}}
but this is potentially not practical, or even desirable.#2019-10-1120:41seancorfield@cjsauer clojure.java.jdbc and next.jdbc have "optional specs" -- they have them in a separate namespace in the project and if you want to use them, you just require the *.specs namespace. That allows the projects to run on Clojure versions older than 1.9 (i.e., before spec was created) but the specs are still bundled with the project/artifact.#2019-10-1120:42alexmiller^^#2019-10-1120:42rapskalianYeah that seems much cleaner#2019-10-1120:43alexmilleror put them in separate libs, but that's a lot more work#2019-10-1120:43seancorfield(well, next.jdbc requires at least Clojure 1.10 because it depends on clojure.datafy but I could probably make that feature optional as well)#2019-10-1120:44rapskalianMy time in web dev forces this desire to keep the dep small in size…but for jars that really doesn’t matter I suppose…
For example, in this datomic ions lib I’m working on, I have an example project included in a separate source folder. I toyed with ideas for not including it in the final artifact, but there’s really no gain from that effort (and it would even prevent me from deploying the example, because ions uses your top-level deps, i.e. it doesn’t take aliases into account)#2019-10-1120:45rapskalianSeems fine to have examples bundled inside of a lib, yeah?#2019-10-1120:47rapskalianNot to mention that the cljs compiler will strip any unused code out anyway. My brain is still catching up to these awesome tools 😅#2019-10-1120:50seancorfieldI'd probably create a separate project for the example stuff that depended on your main project and then just document it copiously in the main project. But, as you say, source is tiny so shipping an example in the main project doesn't hurt as long as it's all in separate namespaces that users can ignore if they aren't interested in it.#2019-10-1120:51seancorfieldAlso, you can have a git dependency that uses a subfolder of the repo -- which might be another was to offer it as an example to folks?#2019-10-1120:52seancorfield:deps/root#2019-10-1120:52rapskalianIt actually was a separate repository, but it made developing the lib such a pain because changes are happening rapidly at the moment. It’s likely something that can be extracted once the API stabilizes tho.#2019-10-1120:52rapskalianOh cool. Is that different than :local/root?#2019-10-1120:53seancorfieldYes, it goes with :git/url to tell t.d.a. to navigate into the specified location within the repo.#2019-10-1120:54rapskalianInteresting…I might have to try that. What is the REPL experience like with this? It’d be great to be able to eval things and have the example pick them up instantly.#2019-10-1120:54seancorfieldSee https://clojure.org/reference/deps_and_cli#_dependencies#2019-10-1120:55seancorfield"It’d be great to be able to eval things and have the example pick them up instantly." -- that is what I would expect with an RDD workflow.#2019-10-1120:57seancorfieldWhen I'm working on a project, I do everything via the REPL. So I either start the apps/services from inside the REPL, or I add the JVM options to start a Socket REPL when the app/service starts up (such as on QA/production). Then I connect my editor to that REPL and eval code as I go, so changes are picked up immediately by the running apps/services.#2019-10-1120:58seancorfieldWe sometimes apply hot fixes to production processes that way -- zero downtime 🙂#2019-10-1120:59rapskalianBold! But let’s say one of your services depends on either a :local/root or deps/root dependency, and you realize you need to make a change in that dep. Can you do that easily without rebooting the REPL?#2019-10-1120:59rapskalianIn Cider I can jump into a lib that isn’t mine part of the current project, and make changes and eval them, and it works, but obviously those aren’t “saved” in the traditional sense#2019-10-1121:01rapskalianI’m sure this becomes much less of an issue once the lib begins to stabilize. At the moment, my example serves as something of a test bed, so changes between the two are happening constantly. The convenience of having both in the same deps project is really nice, but it’d become less useful as the lib changes slowed.#2019-10-1121:03rapskalianWhen they were in separate repos, I found I was having to reboot my REPLs often. There might be a better way tho. Especially with :local/root, which I admittedly haven’t experimented with much.#2019-10-1121:07seancorfieldI'm not quite sure what you're asking. If you evaluate code into the REPL, the changes take effect immediately. So, yeah, if you have the source of the lib you're depending on, you can edit it and evaluate it into your REPL without needing to restart it.#2019-10-1121:10seancorfieldWhen you're developing stuff locally and you're code is still volatile, you can use :local/root to point to the folder the source is in and just edit/eval it into the REPL you're already using for your example project.#2019-10-1121:14rapskalianMaybe this is a limitation of cider. REPLs in cider seem to be “scoped” to the current project. If I open the source file of another folder, I no longer seem to have a REPL connected 🤔#2019-10-1121:25dominicmYou're looking for the sesman related commands#2019-10-1121:27rapskalian🙏#2019-10-1121:27rapskalianAhhh there we go! It let me link the file to an existing REPL session.#2019-10-1121:28rapskalianthis changes everything 😭#2019-10-1121:29rapskalian@U04V70XH6 suddenly I see why my questions make no sense…all this time everyone else has been eval’ing arbitrary source code.#2019-10-1121:35seancorfieldI assume that's to do with CIDER's ability to work with several open REPLs, one for each "project"? I remember when I switched from Emacs/CIDER/nREPL to Atom/ProtoREPL (three years ago) at first I was annoyed that ProtoREPL didn't let me just switch between multiple REPLs but over time I just adopted a workflow where I can work in one REPL across multiple projects -- and now I tend to run a single "everything" REPL for days or even weeks, and just eval in any source code I need to work with.#2019-10-1121:37seancorfield(and I'm now figuring out how best to work with add-lib on the t.d.a. branch so I don't even need to restart a REPL to add new library dependencies)#2019-10-1121:46rapskalianCIDER actually had that functionality way back when, I really miss it. Hot-loading deps was super convenient.#2019-10-1122:05seancorfieldI have a :deps alias with the add-lib branch and then (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
and then (add-lib 'some/new-lib {:mvn/version "RELEASE"}) and it downloads the new library and then it can be required.#2019-10-1122:06seancorfieldWhat I was missing was two lines of code to set up a DynamicClassLoader immediately before starting my Socket REPL and REBL (I have a small ws.dev.repl/-main function for doing that, based on what it finds on the classpath).#2019-10-1122:07seancorfield(I've already opened an issue against REBL to suggest that it starts its REPL in a DCL)#2019-10-1212:57dominicmadd-lib works with cider too fwiw. I added support a little while ago.#2019-10-1121:15seancorfieldHmm, I don't use CIDER.#2019-10-1121:15rapskalianThe chances that I’m doing something wrong are overwhelmingly high tho 😅#2019-10-1121:16seancorfieldI use Atom with Chlorine so I can use simple Socket REPLs with no additional dependencies in my code.#2019-10-1121:17seancorfieldSo, for me, there is "only one REPL" and I can evaluate any and all code into it. I like simple tooling. CIDER is not simple. nREPL is not simple.#2019-10-1121:18seancorfieldStu Halloway was interviewed on the ClojureScript podcast about his REPL usage and he uses really minimal tooling (a plain REPL with inferior mode in Emacs I think he said). I think Rich is the same.#2019-10-1121:20seancorfieldChlorine gives me auto-complete using just the built-in features in Clojure, which I also like (it used to require Compliment for that -- which is also part of CIDER/nREPL).#2019-10-1121:20seancorfieldIt's also why I was so happy to switch from lein/`boot` to the new CLI/`deps.edn` stuff. Simple tooling.#2019-10-1121:26rapskalianHm, I may have to give inf-clojure a try at some point. Plain old eval would likely be just fine honestly. And I had a similar experience moving from lein to deps.edn…breath of fresh air.#2019-10-1514:48murtaza52how do I clear my cache, and force compilation with clj, line what lein clean used to do ?#2019-10-1514:50dominicmclj -Sforce#2019-10-1514:50dominicmOr rm .cpcache#2019-10-1514:55murtaza52.cpcache contains compiled class files ?#2019-10-1515:02andy.fingerhutI believe .cpcache does not contain any compiled class files, but in some cases it will contains names of JAR files, e.g. in your $HOME/.m2 directory.#2019-10-1515:03andy.fingerhutclj commands do not normally save any results from compilation, unless you have some alias that causes evaluation of expressions that write .class files (again, I am pretty sure there, but others will correct if I am missing something).#2019-10-1515:06seancorfield@murtaza52 unless you explicitly AOT code, clj does not produce .class files at all. #2019-10-1515:28murtaza52thanks guys#2019-10-1620:13unbalancedeya... do we have anything like clj -A:new templates yet?#2019-10-1620:50seancorfield@goomba clj -A:new can use all of the Leiningen and Boot templates (they'll just produce whatever they are designed to produce).#2019-10-1620:50unbalancedwhaaaaaaaaaaaaaaaaaaaaaat#2019-10-1620:50seancorfieldBut I doubt there are any clj-specific templates out there (yet).#2019-10-1620:51unbalancedpsssshhhh I made one#2019-10-1620:51unbalancedyou made it easy enough even for a @goomba ™ to figure out#2019-10-1620:52seancorfieldOh, there are some ... https://clojars.org/search?q=clj-template look for /clj-template in the name.#2019-10-1620:52seancorfieldA better search https://clojars.org/search?q=artifact-id%3Aclj-template#2019-10-1620:53unbalancedcool! 😄 (man I was dangerously close to getting "bing"ed again hahaha)#2019-10-1620:54seancorfieldThe main issue is that clj -A:new can pull in templates from git or local deps (which lein/boot cannot) and those would never be discovered by -Sdescribe shows deps.edn at the end of :config-files, and not just the system and user deps?#2019-10-1621:28alexmillerI think it's always been that way#2019-10-1621:29alexmillerI don't remember changing anything about it at least#2019-10-1621:31seancorfield@cfleming It only shows that if you're in a folder that has deps.edn in it.#2019-10-1621:31cflemingYeah, looking at the history of the script looks like it’s always been like that.#2019-10-1621:32cfleming@seancorfield Oh, I see, you’re right.#2019-10-1621:33cfleming@alexmiller that’s the source of the weird bug the other day where stijn was getting a spurious dep - he’s on linux, which starts IntelliJ in his home directory, and he had half a deps.edn file in his home dir.#2019-10-1621:33alexmillerah, cool#2019-10-1621:34alexmillerand by cool, I mean very confusing and not cool#2019-10-1621:34cflemingI think that I usually run -Sdescribe without setting the CWD since I didn’t realise it was significant.#2019-10-1621:34alexmillerhey, it is#2019-10-1621:34cflemingIt’s a weird one, for sure.#2019-10-1621:34cflemingThanks 🙂#2019-10-1621:35cflemingAny chance of ever getting that vector replaced by a map with actual names for the different paths?#2019-10-1621:36cflemingIt’s really ambiguous now - this has also caused confusion around the system file being bundled in t.d.a, too#2019-10-1621:36cflemingI have a feeling I’ve asked for that previously, but I can’t remember the conclusion.#2019-10-1621:50seancorfieldIt needs to be ordered -- that's how the files are merged -- and, for a while, the system one was removed from the list because it got built in (but I think it was restored, at least until tooling catches up with that change?).#2019-10-1621:50seancorfieldWell, it was restored, but I think that's why 🙂#2019-10-1621:52seancorfieldLots of tooling depends on the vector / order so it couldn't change to be a map, but I guess one or more new keys could be added to the overall describe result.#2019-10-1622:07alexmiller"Lots" == ?#2019-10-1622:08cflemingCursive, for one#2019-10-1622:08alexmillerwell that's the one I was aware of :)#2019-10-1622:08alexmillerI was wondering what else was in the "lots"#2019-10-1622:09cflemingCursive doesn’t really depend on the order per se, but does want to understand which of the files are system/user config files and which are the project files.#2019-10-1622:10alexmillerthat should be easier#2019-10-1622:12seancorfieldTrue, since the API changed, there should be fewer-to-none using the describe output now 🙂#2019-10-1622:13seancorfieldI've just seen several tool authors talk about reading that list to get at the raw deps.edn files instead -- although I keep telling them not to do that!#2019-10-1622:14seancorfielddepot just went through a major rewrite to "unify" its read/check behavior with its write behavior (it will update deps.edn to use the latest versions of artifacts).#2019-10-1622:15seancorfieldPrior to the rewrite, the read/check behavior used the built-in t.d.a. logic and looked at the resulting deps/versions -- but the write behavior read deps.edn files directly. Now it does it on both paths which completely broke our usage of the tool 😞#2019-10-1622:17alexmillerI sent PRs to all the projects I could find using clj -Sdescribe output this summer and it actually emits a big ugly warning to stderr if you use it, which I have seen no one mention#2019-10-1622:19alexmillerthis is via the ctda.reader/clojure-env#2019-10-1622:21seancorfieldI think depot just reads the local deps.edn by default or the list of files you provide on the command-line otherwise. I don't think it even respects CLJ_CONFIG.#2019-10-1622:02alexmillerI am actually working on a bunch of changes in this area right now#2019-10-1622:04alexmillerwe are naming the different deps files#2019-10-1622:04alexmillerhow that affects -Sdescribe output tbd (but not planning to break the existing)#2019-10-1622:05alexmillerwe may need to have a conversation @cfleming to make sure you stay in sync, but I need to work through it a little more first#2019-10-1622:07cfleming@seancorfield Right, but you can still apply an ordering to the map elements (i.e. :system comes before :user comes before :project)#2019-10-1622:08alexmilleryes#2019-10-1622:09alexmillerthe ordering can be independent (or more likely, fixed)#2019-10-1622:10cflemingAnyway, I’m glad to hear changes are coming, much appreciated!#2019-10-1622:11alexmillerdo you call anything in clojure.tools.deps.alpha.script.make-classpath ?#2019-10-1622:11cflemingI don’t think so, let me check…#2019-10-1622:13cflemingNo, I use c.t.d.a, c.t.d.a.reader and c.t.d.a.util.dir#2019-10-1622:13alexmillercool#2019-10-1622:14alexmillerthat should all be stable (well, I added ctda.reader/user-deps-location if that's useful)#2019-10-1622:15alexmillerbut I am adding some features that are changing the glue between the clojure script and tda (really all logic in make-classpath)#2019-10-1622:16alexmillerso if you're replicating that logic, it will need to change#2019-10-1622:16alexmillerbut I should really go finish all that so I can get it in rather than talking here :)#2019-10-1622:19cflemingInteresting, thanks. I don’t think that’s relevant to Cursive since IntelliJ doesn’t reliably receive env vars from the shell environment, but I do need to allow users to specify where their user deps.edn file is. I already allow that for users using t.d.a directly, but not for CLI users at the moment.#2019-10-1622:20cflemingSpeaking of which, what’s the state of play of CLI on Windows? I haven’t followed that.#2019-10-1622:21alexmillerwhen you use t.d.a directly, there may be something additional you need to do#2019-10-1622:21alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows is capturing many of the issues#2019-10-1622:21alexmillerI wouldn't say it's done :)#2019-10-1622:22cflemingHehe#2019-10-1622:22cflemingWhat is the additional thing I may need to do when using t.d.a?#2019-10-1622:23seancorfieldFYI, all I use in clj-new is [clojure.tools.deps.alpha :as deps]
[clojure.tools.deps.alpha.reader :refer [default-deps
read-deps]]
[clojure.tools.deps.alpha.util.session :as session]
So I assume I'm safe @alexmiller?#2019-10-1622:23alexmillerlet me finish what I'm working on and then I can have things to point at#2019-10-1622:23seancorfieldHaha... OK.#2019-10-1622:23alexmillerI can't imagine clj-new is going to be affected by any of this#2019-10-1622:24cflemingOk, I shall await it with bated breath…#2019-10-1622:24alexmilleractually it will be able to make use of some new stuff#2019-10-1622:24alexmillerbut I'm going to go work on it :)#2019-10-1714:17holyjakREBL advice needed (is the a better channel?) : I run a remote nrepl server and connect to it with a local nrepl client (mostly via Cursive). I want to include REBL but run it on the local machine, not the server. Is it possible? (I know there is a nrepl middleware for REBL but I guess it assumes REBL is running on the same jvm as the repl itself). Thank you!#2019-10-1719:49Aleksander#rebl#2019-10-1808:21rickmoynihanIt’s not possible. REBL has to exist in the same JVM as the forms it’s browsing. I spoke to Rich about this a while back, and IIRC it’s because to do it you need to implement weak-references over the wire; to enable GC etc. (He implemented such a system years ago in common lisp http://foil.sourceforge.net/ and I guess wanted to avoid the complexity for only marginal use-cases)#2019-10-1808:22rickmoynihanmy nrebl middleware works reasonably well though but it does assume you’re in the same vm:
https://github.com/RickMoynihan/nrebl.middleware#2019-10-1809:27holyjakI see, thank you! 😢#2019-10-1714:20vlaaadthere is #rebl#2019-10-1714:23vlaaadI played with rebl, can't tell anything about nrepl, but rebl uses prepl, and you can pass your own prepl function to it#2019-10-1714:24vlaaadcognitect.rebl/ui accepts :proc keyword argument, which should be a function of 2 args: in-reader and out-fn, as clojure.core.server/prepl#2019-10-1714:25vlaaadthere is clore.core.server/remote-prepl, which can be transformed to satisfy :proc#2019-10-1714:26vlaaadI played with it, here is what worked for me:
(cognitect.rebl/ui :proc
#(clojure.core.server/remote-prepl
"localhost" 50505
%1 %2
:valf (partial clojure.edn/read-string {:default tagged-literal})))
#2019-10-1808:45rickmoynihanI’m pretty sure this approach won’t work properly, especially wrt the datafy/nav behaviours.
https://clojurians.slack.com/archives/C6QH853H8/p1571386900140400?thread_ts=1571321859.133900&cid=C6QH853H8#2019-10-1808:45rickmoynihanthough it may well be good enough for basic use#2019-10-1808:46rickmoynihanbut you’ll not get the metadata/protocol dispatch that REBL supports. REBL will only be able to walk the literal syntactic forms.#2019-10-1808:46vlaaadyes, datafy/nav will be much more limited since it happens in the same vm and not over wire, but rebl will still work.#2019-10-1808:48rickmoynihanAlso it won’t be able to follow the repl etc#2019-10-1808:58vlaaadwhat do you mean follow the repl?#2019-10-1808:59rickmoynihanforms evaled into nrepl won’t appear in REBL#2019-10-1808:59rickmoynihanso only forms entered through REBL itself#2019-10-1808:59rickmoynihanunless i’m misunderstanding something#2019-10-1809:03vlaaadah, yeah, I guess so, because rebl is a socket repl, and nrepl is not#2019-10-1809:06rickmoynihanI don’t think it’s anything to do with the difference between socket-repl and nrepl. I think it’s more to do with that they’re in different processes and can’t share the stdin stream to intercept.#2019-10-1714:27vlaaad(note, I used tagged-literal to be able to read what clojure can't usually read, for example clojurescript's js object notation: #js{:a 1})#2019-10-1810:00timovanderkampHi, I have a dependency that suddenly broke. I am using https://clojars.org/amazonica and it results in the following error: Unable to resolve com.amazonaws/amazon-kinesis-client version: [1.7.5, 1.8.0). Any ideas what couldve caused this?#2019-10-1810:09markwoodhallWe have had the same issue today, it looks like maven-metadata.xml only has one version available now. https://repo1.maven.org/maven2/com/amazonaws/amazon-kinesis-client/maven-metadata.xml, I have asked about it here - https://github.com/awslabs/amazon-kinesis-client/issues/631 but I'm not sure I am asking the right thing.
I fixed temporarily by doing:
wget
wget
mvn install:install-file -Dfile=amazon-kinesis-client-1.7.5.jar -DgroupId=com.amazonaws -DartifactId=amazon-kinesis-client -Dversion=1.7.5 -Dpackaging=jar -Dpomfile=amazon-kinesis-client-1.7.5.pom.xm
#2019-10-1810:13timovanderkampThank you!#2019-10-1812:54alexmillerThis may really need to be filed with sonatype for maven central #2019-10-1812:55alexmillerhttps://issues.sonatype.org#2019-10-1811:32rickmoynihanIs there a way to run clojure -Srepro -Sdeps {,,,} with a flag that prevents it merging the command line deps with a deps.edn file in the working directory?#2019-10-1811:33rickmoynihani.e. an equivalent of Srepro but for the pwd’s deps.edn#2019-10-1812:51alexmillerNo, but I am working on a release right now that may help you. What’s the use case? #2019-10-1812:58alexmillerIs it running a tool?#2019-10-1813:23rickmoynihanHey… Yes it’s running a clj dev tool we have.#2019-10-1813:25rickmoynihanessentially because it merges with deps.edn we end up running into this tdeps issue: https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-101?filter=allissues&orderby=priority%20DESC&keyword=101 as our deps.edn specifies a :mvn/repos
which makes our CI 50s slower than it needs to be#2019-10-1813:25rickmoynihanwe pull in the tool over a git dep#2019-10-1813:27rickmoynihanhow might the new release help?#2019-10-1813:41alexmillerthe new feature lets an alias run with isolated rather than merged :deps with the project deps. that's similar but different than what you're wanting#2019-10-1813:41alexmillerdo you want to completely remove the repo or just tweak it?#2019-10-1813:42alexmillerone thing I did happen to add in latest (not yet released) is the ability to remove a repo by setting it's name to nil#2019-10-1813:43alexmillerso if that was sufficient you could -Sdeps '{:mvn/repos {"the-server" nil}}'#2019-10-1814:37rickmoynihancompletely removing would be fine#2019-10-1814:38rickmoynihanhmm interesting… that might work#2019-10-1814:38rickmoynihanI didn’t think of that#2019-10-1814:39alexmillerthat will fail on current release, but should work on next release, coming. ... hopefully real soon#2019-10-1814:39rickmoynihanawesome — is real soon likely to be some time today? or next few days?#2019-10-1814:40rickmoynihanno need to promise 🙂#2019-10-1814:40alexmillerhopefully today, although it sometimes takes a day for the brew pr to make it through#2019-10-1814:40rickmoynihanyeah stuff usually takes longer than you think#2019-10-1814:41rickmoynihanAnyway, thanks again for this… it’s very timely.#2019-10-1814:42rickmoynihanisolated deps would also be useful too — I’ve wanted that from time to time#2019-10-1814:42rickmoynihanbut instead settled for refactoring stuff out of the core deps into aliases — the problem being the default case then suffers with extra alias baggage#2019-10-1814:43alexmilleryeah, it is particularly useful for tools#2019-10-1815:37zetafishIs there a way to prevent tools-deps to download maven-metadata.xml files?#2019-10-1815:55alexmillercurrently, no.#2019-10-1815:55alexmillerare you looking for an "offline" mode?#2019-10-1815:57alexmillerif you already have needed versions locally, I'm actually not sure off the top of my head whether it will make any remote calls.#2019-10-1815:57zetafishIt is because of the https://github.com/awslabs/amazon-kinesis-client/issues/631#2019-10-1815:58alexmilleryeah, I don't think there's any way to do explicitly that right now#2019-10-1815:59zetafishI am using amazonica in deps.edn and explicitly exclusions the kinisis-client-library. It all worked until today that is. The annoying thing is that I do not even want to use the kinesis-client-library.#2019-10-1816:00alexmillerexclusions work on the library level, not version level. is it just running into trying to get an older version to do the check?#2019-10-1816:01zetafishSnippet of deps.edn:
com.amazonaws/aws-java-sdk-core {:mvn/version "1.11.521"}
com.amazonaws/aws-java-sdk-sts {:mvn/version "1.11.521"}
com.amazonaws/aws-java-sdk-s3 {:mvn/version "1.11.521"}
amazonica {:mvn/version "0.3.140"
:exclusions
[com.amazonaws/aws-java-sdk
com.amazonaws/amazon-kinesis-client]}
#2019-10-1816:02alexmilleryeah, I was more wondering how it manifests in a clj failure?#2019-10-1816:03zetafish% clj -Stree
Error building classpath. Unable to resolve com.amazonaws/amazon-kinesis-client version: [1.7.5, 1.8.0)
#2019-10-1816:03alexmillerI'm guessing amazonica relies on some particular older version and it's trying to find that, before it gets to the point of checking exclusions#2019-10-1816:03alexmilleryeah#2019-10-1816:03zetafishSo something thinks that it should find [1.7.5, 1.8.0) and then later it is excluded anyway.#2019-10-1816:03alexmilleryeah#2019-10-1816:04alexmillerif you're just looking for a temporary workaround, you may be able to add it at the top level, with the version that's available, and exclude it there too#2019-10-1816:04alexmillercom.amazonaws/amazon-kinesis-client {:mvn/version "1.12.0" :exclusions [com.amazonaws/amazon-kinesis-client]}#2019-10-1816:05zetafishThat looks really silly but let me try it 🙂#2019-10-1816:06alexmillernah, exclusions works only on children#2019-10-1816:06alexmillerso won't work#2019-10-1816:07alexmillerthis will work...#2019-10-1816:07alexmillerclj -Sdeps '{:aliases {:x {:override-deps {com.amazonaws/amazon-kinesis-client {:mvn/version "1.12.0"}}}}}' -A:x#2019-10-1816:07alexmillerbasically supply an alias that overrides the version to use to the one that happens to be available#2019-10-1816:09alexmilleractually, never mind, you'll still hit this#2019-10-1816:09zetafishSadly no#2019-10-1816:10alexmillerhave you checked any of hte maven mirrors?#2019-10-1816:11zetafishNot yet#2019-10-1816:12zetafishBut I guess these mirrors will be updated to central at a certain moment, as they are mirrors.#2019-10-1816:12alexmillermaybe#2019-10-1816:12zetafishOr does each mirror keeps their owne maven-metadata#2019-10-1816:13alexmillerhttp://uk.maven.org/maven2/com/amazonaws/amazon-kinesis-client/ seems to have them#2019-10-1816:13alexmilleroh, not in metadata though#2019-10-1816:15alexmillerthe google mirror seems to have them https://maven-central.storage-download.googleapis.com/repos/central/data/com/amazonaws/amazon-kinesis-client/maven-metadata.xml#2019-10-1816:16alexmillerso you could try adding {:mvn/repos {"central" {:url ""}}}#2019-10-1816:18alexmillerthat doesn't seem to work for me, but I'm not sure why#2019-10-1816:20zetafishThanks. That will probably work (until it gets hit by 1.12.0 as well)#2019-10-1816:27alexmilleroh, that works if I rm the amazon-kinesis-client out of my ~/.m2 (it caches that source metadata)#2019-10-1816:27alexmillermaven central uploads have been giving me many weird errors over the last 24 hrs#2019-10-1816:28alexmillermight be something is awry there and an indexing job failed#2019-10-1816:28alexmillerdid you file a ticket with sonatype?#2019-10-1816:28alexmillerI have an acct there already and I can do so if needed#2019-10-1817:16zetafishIt would be great if you filed a ticket there.#2019-10-1819:13alexmillerhttps://issues.sonatype.org/browse/MVNCENTRAL-5283#2019-10-2113:19alexmillerlooks like it's fixed now#2019-10-1817:34rickmoynihanour CI (travis) has .m2 cached… but it seems that running clojure for the first time on a single private git/dep in each build container refetches these deps each time:
Downloading: org/clojure/clojure/1.10.0/clojure-1.10.0.pom from
Checking out:
#2019-10-1817:35rickmoynihanI don’t know where they’re coming from… as hiccup is not a dependency of omni; and they already exist in .m2#2019-10-1817:36rickmoynihanso it looks like tools.deps is refetching them#2019-10-1817:42alexmillerI can't really do anything without more info. need 1) deps.edn and 2) output of clj -Stree -Sdeps '{:aliases {:v {:verbose true}}}' -A:v#2019-10-1817:42alexmillerI can't really do anything without more info. need 1) deps.edn and 2) output of clj -Stree -Sdeps '{:aliases {:v {:verbose true}}}' -A:v#2019-10-1819:03alexmilleris this in response to request on @U06HHF230 - are you working together? or are there two things here?#2019-10-1913:10zetafishWrong me#2019-10-2110:02rickmoynihanNot working together no.
I tried running with :verbose and it looks like the deps are coming from the git dep after all so I suspect this is another instance of not being able to configure updatePolicy on :mvn/repos, i.e. it looks like clojars is somehow using an updatePolicy of daily rather than never.
i.e. I think it’s this issue: https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-101?filter=allissues&orderby=priority%20DESC&keyword=101#2019-10-2110:08rickmoynihanIt’d be nice to support this, as it prohibits caching the deps in CI — which slows down the builds#2019-10-2112:44alexmillerDaily is the default so that matches what I would expect. Are any of these Snapshots?#2019-10-1905:32seancorfieldseancorfield/depstar "0.3.4" -- now follows symlinks when copying directories (thanks to @mgrbyte for reporting the problem)#2019-10-2021:05andy.fingerhutThis is probably too far off the beaten path to matter to anyone, but I noticed recently that Homebrew is also supported on Linux and Windows with WSL, and so tried out 'brew install clojure' on a Linux system. It seems to set things up in a way that is problematic for clojure finding the system-wide deps.edn file. I know there is the clojure install script for Linux, which is what I have always used before today, and will continue to do so. Just wanted to point this out in case anyone else tries it.#2019-10-2022:37alexmillerWhat’s the actual problem you saw? clojure actually no longer needs that file#2019-10-2022:45andy.fingerhutAfter brew install clojure completed with no apparent problems, I tried clojure -Sdescribe and got this message, which at the time stopped me from further experimentation: $ clojure -Sdescribe
cp: cannot stat 'PREFIX/example-deps.edn': No such file or directory
but I can try other things if you think it would be useful to know.#2019-10-2022:47andy.fingerhutA kind of unusual thing I noted about the results after brew install clojure completed was that Homebrew creates a /home/linuxbrew/.linuxbrew directory, where linuxbrew is a new directory, not the name of any user account on the machine, and puts all (or at least the ones I have seen) brew-installed files inside of that dir.#2019-10-2022:49andy.fingerhutComparing to my works-better macOS install via brew install clojure earlier, it appears that the Linux clojure shell script has one occurrence of PREFIX that is not replaced with the install directory.#2019-10-2023:54andy.fingerhutI see what happened -- I tried this on an Ubuntu Linux system with minimal install, and it had no ruby executable installed. brew install clojure's install.sh script uses ruby to replace occurrences of the string PREFIX with the install dir. That step of install.sh script failed, leaving the clojure shell script in its original unmodified form.#2019-10-2023:54andy.fingerhutsudo apt-get install ruby, brew uninstall clojure, then brew install clojure made the install go better.#2019-10-2023:55andy.fingerhutStrange that the Homebrew install went well without installing ruby itself.#2019-10-2023:57seancorfieldI thought brew used Ruby to do pretty much everything?#2019-10-2023:58seancorfieldPerhaps the clojure recipe needs to be updated to explicitly state ruby is a dependency?#2019-10-2023:58seancorfield(I assume brew can be used to install ruby?)#2019-10-2100:02andy.fingerhutI think it can. Homebrew's first platform was macOS, I believe, where ruby has been installed by default since a long time ago (although I hear that might be changing in the latest macOS just released? Or perhaps the next one?)#2019-10-2100:04andy.fingerhutAnd probably if I had done a non-minimal install of Ubuntu 19.04 Linux, it might have included ruby as well. In any case, sudo apt-get install ruby definitely works if you do that before brew install clojure. It seems that the main Homebrew install instructions for Linux work without ruby being installed first, surprisingly enough.#2019-10-2100:04seancorfieldI just tried install Homebrew on WSL based on https://docs.brew.sh/Homebrew-on-Linux and the first thing it did was install Ruby...#2019-10-2100:05seancorfield(but it might be an "embedded" Ruby for its own use?)#2019-10-2100:05andy.fingerhutRuby got installed after you did only this command? sh -c "$(curl -fsSL )" ?#2019-10-2100:07andy.fingerhutI will test on a fresh Linux VM and look at what happens a little more carefully while installing Homebrew.#2019-10-2100:07seancorfieldYeah, just confirmed it's an internal version of Ruby, just to run Homebrew! 😐#2019-10-2100:09seancorfieldHmm, and to get Ruby you need patchelf which in turn requires Clang / gcc... OK, not going down that rabbit hole! 🙂#2019-10-2100:09seancorfieldSo, could Clojure's brew recipe use something other than Ruby to do that textual change in the script?#2019-10-2100:10andy.fingerhutEven a minimal Ubuntu 19.10 Linux install includes gcc, and I remember brew installation installed patchelf early on before.#2019-10-2100:11seancorfieldThis is an earlier Ubuntu from an early WSL1 install (the joys of Insider builds)#2019-10-2100:13seancorfield(This is on my Mac running Win10 via Parallels with WSL as just a testing ground... maybe later I'll try it on my Win10 laptop although that's also WSL1 but I use it a lot more there... on this machine, I use Mac's Terminal for all my work)#2019-10-2100:15seancorfieldAlthough on Windows now I'm using Scoop to install Clojure etc in PowerShell#2019-10-2100:17andy.fingerhutI believe I have followed the Homebrew Linux install instructions faithfully, and while it does install a ruby executable, it seems to be in a directory that does not end up on your command PATH after following the instructions, but some other directory.#2019-10-2100:17seancorfieldRight:
> Yeah, just confirmed it's an internal version of Ruby, just to run Homebrew!#2019-10-2100:20andy.fingerhutIt seems that perhaps there ought to be a way (or perhaps already is?) for a Homebrew install script to access that internal version.#2019-10-2100:20andy.fingerhutbut that's a minor issue of polish, I think.#2019-10-2100:22seancorfieldThis looks like the only line that is problematic: https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/install.sh#L14 -- if that used any low-level, guaranteed-to-be-installed command to change the text, it wouldn't need Ruby and it would work just fine.#2019-10-2100:22seancorfieldsed perhaps?#2019-10-2100:37andy.fingerhutPerhaps. It does appear that sed has a -i option on the couple of systems where I have checked, for easier overwrite-the-input-file-in-one-shell-command convenience.#2019-10-2101:01seancorfieldGiven that there's a cp just below it, why not use this directly sed "s;PREFIX;$prefix;" < clojure > $prefix/bin/clojure#2019-10-2101:02seancorfieldSo it would be mkdir ...
sed ...
cp ...
#2019-10-2101:04seancorfieldI don't see much reason to modify the installed source files in place.#2019-10-2101:05seancorfield(I'll be interested to hear Alex's thoughts on this tomorrow)#2019-10-2103:21alexmillersed has incompatible syntax between Mac and Linux #2019-10-2108:04dominicmIt's the -i flag that's incompatible. You can easily avoid it.#2019-10-2103:22alexmillerIf you can make it work, patch wanted#2019-10-2104:59andy.fingerhutAny doc page you would recommend on how to test modifications to a Homebrew install script?#2019-10-2105:10andy.fingerhutI have tested on a Linux machine using brew install --interactive clojure where I edited the install.sh script to replace ruby with ${HOMEBREW_RUBY_PATH}, and that worked. It used the Homebrew-internal installation of ruby. I will try that on a macOS system next to see if it works there, too.#2019-10-2105:14andy.fingerhutYep, worked on a macOS system, too. That shell variable ${HOMEBREW_RUBY_PATH} was the full path name to a Homebrew-internal ruby executable on both a Linux system that had no other ruby installed, as well as on a Mac that had /usr/bin/ruby preinstalled before Homebrew was installed.#2019-10-2105:21andy.fingerhutI created a PR, even though they are likely not accepted for this project, just one convenient way to show the suggested change: https://github.com/clojure/brew-install/pull/5 . Feel free to close it.#2019-10-2105:25hiredmanI think "brew-install" in the project name is a misnomer, and install.sh is expected to work on Linux sans homebrew#2019-10-2105:26andy.fingerhutThe install.sh script that my PR suggests changing appears to be identical to a file with the same name that I found in both a Mac and Linux Homebrew installation when I installed the clojure Flask.#2019-10-2105:28andy.fingerhutThe line my PR suggests changing is the one that was failing on a Linux system with no ruby executable.#2019-10-2105:29andy.fingerhutI mean, I am fairly new to this Homebrew stuff, so could be making any number of mistaken assumptions here, but giving it a shot.#2019-10-2105:29hiredmanSure, and it is also, unless I am mistaken, identical to install.sh which is used for installing on Linux systems which do not use homebrew#2019-10-2105:30andy.fingerhutThere is another file named linux-install.sh in the same directory of that repo with install.sh, which uses install commands, that I suspect might be the non-Homebrew Linux install script.#2019-10-2105:30hiredmanAh, my mistake#2019-10-2105:32andy.fingerhutAlways glad for another mind on the problem, though. Thanks for looking into it.#2019-10-2105:39hiredmanI use Linux as my desktop os, and the idea of using homebrew to manage installed packages is so alien to me. I would consider just making the homebrew script exit with an error suggesting using the Linux install script on Linux. In which case you could depend on the bsd sed options and wouldn't need Ruby. But that is maybe a terribly unfriendly idea.#2019-10-2105:41andy.fingerhutThe script works fine on macOS already today, since /usr/bin/ruby comes preinstalled (until either Catalina, I think, or maybe the next macOS release, when I have read it will no longer come preinstalled). With this change, it should continue to work on macOS, and start working on Linux. That doesn't mean anyone must use Homebrew on Linux -- just another option.#2019-10-2108:05dominicmI would just not sed in place, and use it to write elsewhere.#2019-10-2108:05dominicm-i is just a convenience flag with sed#2019-10-2109:11tianshuhow to add a repository in deps.edn?#2019-10-2109:25dominicmThere's a :mvn/repos key#2019-10-2109:56sogaiuhttps://clojure.org/reference/deps_and_cli#_procurers ?#2019-10-2110:01tianshu@dominicm @sogaiu thanks!#2019-10-2110:22kommensince upgrading to clojure 1.10.1.478 (from https://github.com/Homebrew/homebrew-core/pull/45565) clojure fails to run#2019-10-2110:23kommen$ clojure
Error: Could not find or load main class clojure.main
$ clojure -Spath
Error: Could not find or load main class clojure.main
#2019-10-2110:23kommen$ brew info clojure
clojure: stable 1.10.1.478
The Clojure Programming Language
/usr/local/Cellar/clojure/1.10.1.478 (9 files, 21.1MB) *
Built from source on 2019-10-19 at 20:39:17
From:
==> Dependencies
Required: rlwrap ✔
==> Requirements
Required: java >= 1.8 ✔
==> Analytics
install: 3,096 (30 days), 11,637 (90 days), 49,842 (365 days)
install_on_request: 3,070 (30 days), 11,511 (90 days), 49,114 (365 days)
build_error: 0 (30 days)
#2019-10-2110:33kommenany body else having issues like this?#2019-10-2110:34dominicmHave you wiped your .m2?#2019-10-2110:34dominicm(I'm not suggesting you should)#2019-10-2110:38kommenno, I didn’t#2019-10-2110:39kommenI just tried to move .clojure/deps.edn somewhere else, suspecting a problem with it. but then I get:#2019-10-2110:39kommen$ clj
cp: PREFIX/example-deps.edn: No such file or directory
#2019-10-2110:39kommenwith an ~/.clojure/deps.edn file only containing {} I guet the same Error: Could not find or load main class clojure.main error#2019-10-2110:39dominicmAh.#2019-10-2110:48kommensince the -m flag of the java call was changed in this release, I suspect that this would be the problem:#2019-10-2110:48kommenhttps://github.com/clojure/brew-install/commit/0accd10be691101e6288e1bf614abe7371b43cc2#2019-10-2111:01dominicmI don't think it is. I think your install failed.#2019-10-2111:02dominicmGiven that you have issues with PREFIX#2019-10-2111:03kommenyes, you’re right#2019-10-2111:03kommentried to reinstall already, but no luck#2019-10-2111:04dominicmRuby powers the rewrite on Mac, that's all I know I'm afraid. Maybe there's a log somewhere?#2019-10-2111:04kommenwoudn’t know where to look for a log#2019-10-2111:05dominicmIt'll be a brew install log I'd guess.#2019-10-2111:05kommenso this is supposed to be replaced at time of brew install? https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/clojure#L175#2019-10-2111:05dominicmYes#2019-10-2111:07kommenI see: https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/install.sh#L14#2019-10-2111:13kommenok, so I manually fixed the install_dir= in /usr/local/bin/clojure and it works now, let’s see if this is only a problem with my machine or if some body else runs into it#2019-10-2111:25kommenfor future reference: macOS 10.15 (19A583), $ ruby --version
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15]
brew config
HOMEBREW_VERSION: 2.1.15
#2019-10-2111:36kommenok, found the logs and this also explains it:#2019-10-2111:36kommen$ cat ~/Library/Logs/Homebrew/clojure/01.install.sh
2019-10-21 12:50:08 +0200
./install.sh
/usr/local/Cellar/clojure/1.10.1.478
truffleruby: the -p option is not implemented
#2019-10-2111:48dominicmHaha. Yes it does :)#2019-10-2111:57kommen@dominicm thanks for keeping company and pointing me in the right direction#2019-10-2112:49alexmillerOk, so is there something to fix here or is this an issue mainly on your system?#2019-10-2113:01dominicmThe problem is that truffle ruby is installed, and that didn't support -p for doing the substitution. Anyone who's playing with graal may run into this.#2019-10-2113:01alexmillerso the prior discussion here about using the brew install ruby impl would presumably also address that#2019-10-2113:02dominicmYes. So would switching to sed.#2019-10-2113:02alexmiller-i has different incompatible syntax on mac and linux there#2019-10-2113:02alexmillerI used to use sed, then perl for a bit#2019-10-2113:03dominicmI know. But you can avoid using -i#2019-10-2113:03alexmillerI seem to remember there was some reason I didn't do that either, but it was a long time ago now#2019-10-2113:08dominicmProbably not a problem. Using brew means ruby anyway. The only impetus would be to unify the install scripts.#2019-10-2113:11plexusunfortunately some distros have stopped including ed... echo -e 'w clojure.bak\n1,$s^PREFIX^'$PREFIX'^\nwq' | ed clojure#2019-10-2113:41kommenI remember I once proposed a sed command which worked on linux and macos, but either my mind or jira search fails me#2019-10-2113:43andy.fingerhutThis is exactly the same problem I found on Linux with Homebrew with no Ruby preinstalled. This will be an issue for all macOS Catalina systems, too, which also have no ruby preinstalled for the first time with macOS since it was introduced almost 20 years ago.#2019-10-2113:43kommenbut Dominic is right, using brew means using ruby, so I guess it makes sense to be able to rely on ruby#2019-10-2113:43andy.fingerhutThere is an issue open for Homebrew for this since June 2019 I found last night.#2019-10-2113:44andy.fingerhutI believe that the patch I submitted for the Clojure Homebrew installer will work for Catalina as well, because it will force the Clojure install.sh to use a ruby installed inside of Homebrew, rather than a system-wide one.#2019-10-2113:44andy.fingerhutBut I do not have a macOS Catalina system to test on to be certain.#2019-10-2113:46alexmillerwell, that certainly bumps it up my priority list#2019-10-2114:55markbastianDoes anyone know if the immutant folks intended to let their domain expire? When I go to http://immutant.org/ there's just a domain expiration notification.#2019-10-2115:05dominicmThis probably isn't the channel for that. @U06SGCEHJ you're associated I think?#2019-10-2115:37alexmillerdefinitely not the channel, but I'm not sure anyone is still working on immutant#2019-10-2115:40markbastianOk, thanks for the info. I'll see if there's a better place.#2019-10-2115:43markbastianOh, I just noticed I posted this in tools-deps, not clojure. Sorry!#2019-10-2115:55alexmillertools.deps.alpha 0.8.578, clj 1.10.1.478 are now available - see #announcements for details#2019-10-2115:58alexmilleras a side note, there were changes made to the internal cli scripts between clj and tools.deps (clojure.tools.deps.alpha.script.make-classpath and clojure.tools.deps.alpha.script.generate-manifest). I doubt many people use any of that directly, but changes were made in clojure.tools.deps.alpha.script.make-classpath2 and clojure.tools.deps.alpha.script.generate-manifest2 to avoid any breakage.#2019-10-2115:58alexmillerno changes to tda apis#2019-10-2122:13ro6Is anyone successfully using (add-lib ...) in a Cursive repl? I get a true back, but then "could not locate ... on classpath" when requiring the lib.#2019-10-2122:17cfleming@alexmiller The new :deps and :paths support in aliases is interesting. Am I correct in thinking that this would allow overriding the system deps.edn, e.g. to use a fork of Clojure?#2019-10-2122:18alexmillerIt still has install deps on the chain#2019-10-2122:18alexmillerSo, no difference there#2019-10-2122:19cflemingI see. Yes, re-reading the announcement I see I misread it - it’s more for tools to not require the project classpath instead.#2019-10-2122:19cflemingThanks.#2019-10-2206:33dominicmOoh, that would be very useful for pack#2019-10-2207:11andy.fingerhutCorrection to something I said earlier -- macOS Catalina (the recently released version, earlier this month) does still include /usr/bin/ruby, perl, and python executables installed by default. I checked by installing it on one of my Macs in the house. Apple has announced that the next major release will not include them in the base macOS installation.#2019-10-2207:15andy.fingerhutI will soon be installing Homebrew on it, and brew install clojure, to see if it works for me.#2019-10-2208:40andy.fingerhutAfter installing a Java executable (I chose AdoptOpenJDK 11) and Xcode, and Homebrew, brew install clojure produced a working clojure command install on my macOS Catalina system. There is no need that I can see to change the install.sh script to make it work on that OS.#2019-10-2208:41andy.fingerhutIt isn't clear to me why someone earlier had a problem there, unless perhaps /usr/bin wasn't in their path so that /usr/bin/ruby was not found, but that seems unlikely to be the reason.#2019-10-2213:53kommen@andy.fingerhut if you mean me by someone earlier with macOS catalina: this was due to truffleruby (from graalvm) being used to run the install.sh script. and truffle ruby doesn’t support the -p flag yet: https://clojurians.slack.com/archives/C6QH853H8/p1571657798259600#2019-10-2213:57andy.fingerhutSo at the time you tried upgrading the clojure package using brew upgrade clojure, you had a ruby executable in your command PATH that was earlier than /usr/bin/ruby? If so, the results you got makes sense to me.#2019-10-2306:50kommen@andy.fingerhut no, that is the part I didn’t understand but also didn’t investigate yet: how brew install clojure came to use truffleruby. truffleruby is not on my PATH…#2019-10-2312:11delaguardoAre you using graalvm with ruby component installed?#2019-10-2312:13kommenyes#2019-10-2313:02delaguardoprobably you have graalvm bin directory in your path that overrides /usr/bin#2019-10-2313:03kommenno, that isn’t the case. also running ruby --version in the same shell as the brew commands shows that it uses a ruby version managed by rbenv#2019-10-2313:39delaguardothat is right, but during formula installation it loads new shell session with limited number of tools installed. The best way to check it - brew install --interactive clojure#2019-10-2313:40delaguardoAs an example:
Install to this prefix: /usr/local/Cellar/clojure/1.10.1.478
/Users/delaguardo/.zshrc:29: command not found: starship
/Users/delaguardo/.zshrc:32: command not found: jenv
/Users/delaguardo/.zshrc:49: command not found: rbenv
clojure-tools % which ruby
/usr/bin/ruby
clojure-tools % ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
#2019-10-2313:42andy.fingerhutIn any case, I saw that Alex Miller has committed a change that I suggested where brew install clojure will run a shell script named install.sh such that it will run ruby using ${HOMEBREW_RUBY_PATH}, which Homebrew should consistently set to the full path name of its own internal ruby executable. This should make brew install clojure much more robust in the face of arbitrary user environments.#2019-10-2313:42andy.fingerhuthttps://github.com/clojure/brew-install/commit/f238dcebaa0b7bce51305c322dfb89fd3e152304#2019-10-2313:42andy.fingerhutI don't think that commit is deployed "to the field" yet, but on its way in that direction.#2019-10-2313:44kommenthanks @U04V4KLKC, that would explain it. I will check later#2019-10-2313:46kommen@andy.fingerhut thanks for the info, I’ll make sure to test brew install clojure with the next release#2019-10-2313:46delaguardo> In any case, I saw that Alex Miller has committed a change that I suggested where brew install clojure will run a shell script named install.sh such that it will run ruby using ${HOMEBREW_RUBY_PATH}, which Homebrew should consistently set to the full path name of its own internal ruby executable. This should make brew install clojure much more robust in the face of arbitrary user environments.
I’m not sure that this is the best solution. Now brew-install repo has two almost identical scripts that are doing the same thing but differently (using different tools)#2019-10-2313:47delaguardorecently I did some changes - https://github.com/clojure/brew-install/compare/1.10.1...DeLaGuardo:1.10.1
to create one cross-platform installation script#2019-10-2313:51delaguardoNot in PR yet because I was hoping to find some time to build packages for other distribution platforms like debian’s ppa and rpm for centos#2019-10-2405:47amarjeetHi, is there a way to provide credentials while using private git repo as a dependency in deps.edn? I want to use my private git repos in my project, but I am getting credentials related error.#2019-10-2412:07alexmillerWhat kind of credentials?#2019-10-2412:07alexmillerssh is supported, user/pw is not currently#2019-10-2414:51amarjeet@U064X3EF3 For ssh, should I supply an additional key as :ssh in the deps map, in addition to :git/url and :sha?#2019-10-2415:03alexmillerno#2019-10-2415:03alexmillerwe use the ssh-agent as a source for identity checking#2019-10-2415:04amarjeetoh okay#2019-10-2415:04alexmillersee https://clojure.org/reference/deps_and_cli and search for "Git configuration" for more info#2019-10-2415:05alexmilleras a general principle, you will never put any kind of creds in deps.edn#2019-10-2415:05amarjeetGot it, makes sense. I will do it and try. Thanks much 🙂#2019-10-2510:13borkdudecan I specify a src folder for a local/root with a pom.xml (it's a boot project)#2019-10-2510:14borkdudesomehow the source path doesn't show up in the pom.xml and when I run clojure -Spath it returns src/main/clojure#2019-10-2510:14borkdudethe library is called dre.common and this is on my classpath: /Users/borkdude/dre/DocSearch/common/src/main/clojure#2019-10-2510:40borkdudeAdding "../common/src" to the extra-paths works, but not sure if that's the right way#2019-10-2512:30delaguardoI have some problem trying to reposition library coordinates:
Library A’s deps.edn file
{:deps {B {:mvn/version "1"}}}
Library B’s deps.edn file
{:deps {C {:mvn/version "1"}}}
Library C’s deps.edn file
{:deps {}}
All version exists and can be resolved
Then I want to use git version of C library in A
Updated deps.edn for lib A:
{:deps {B {:mvn/version "1"}
C {:git/url ""
:sha "commit-sha"}}}
I’m expecting git version to take precedence in resolved dependency tree.
Instead I got an error:
Error building classpath. Unable to compare versions for C: {:mvn/version "1", :deps/manifest :mvn} and {:git/url "", :sha "commit-sha", :deps/manifest :deps, :deps/root "/home/user/.gitlibs/libs/C/commit-sha"}
Looks like a bug in clojure-cli for me. Happy to file a ticket if anybody can confirm (or agree) that this is a bug#2019-10-2512:44alexmillerthere is a ticket already I think for being unable to compare versions for the same library across different coordinate types#2019-10-2512:44alexmillerstill a todo#2019-10-2512:45alexmillerI have not read the above carefully, just going by the error#2019-10-2512:49delaguardoTo summarize all above:
main problem that I’m trying to override dependency coordinate by adding it in top-level dep.edn. That is working fine within the same coordinate type (maven and git) but fails when I need git dependency to overtake maven. So yes, I think adding ability to compare different coordinate types should solve it#2019-10-2512:54alexmilleryes, this has not yet been implemented#2019-10-2512:54alexmilleralthough I've thought about it a lot and know what to do#2019-10-2512:59delaguardonice! thanks#2019-10-2512:59delaguardoalso I’m happy to help if you don’t mind to share you ideas around it#2019-10-2513:55gphilippNot sure if it’s of any interest to anybody, but I’ve this small one-liner which pretty-print an EDN form in the clipboard:
clojure -Sdeps '{:deps {mvxcvi/puget {:mvn/version "1.2.0"}}}' -e "(require '[puget.printer :as puget]) (puget/cprint (read-string (first *command-line-args*)))" $(mktemp) "$(pbpaste)"
I’m not sure why the first arg needs to be a file since I’m using the -e option (maybe https://clojure.atlassian.net/browse/TDEPS-56) but well, it works.#2019-10-2514:46borkdude@U050S183X if you're interested in a tool with milliseconds startup time:
$ jet --pretty <<< "$(pbpaste)"
{:a 1, :b 2, :c 3}
#2019-10-2514:55gphilippNice ! Is there an option for color highlighting, as this is why I’m using the command above ? For speed, I use https://github.com/kkinnear/zprint/blob/master/doc/filter.md#appcds-version with echo pbpaste | zprint, it’s also compiled with GraalVM.#2019-10-2514:56borkdudejet doesn't have colors yet.#2019-10-2518:38andy.fingerhutIs there some kind of option to clj/clojure commands to cause them to show the java commands they execute?#2019-10-2519:06seancorfieldbash -x clojure <args>#2019-10-2519:07seancorfield(that shows "everything" but it does show the java command as well 🙂 )#2019-10-2519:07seancorfieldbash -x clojure
...
+ exec /usr/bin/java -Dclojure.libfile=.cpcache/1562564545.libs -classpath src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar clojure.main
Clojure 1.10.1
user=>
#2019-10-2519:08seancorfield(you can't do bash -x clj ... because it doesn't pass -x into clojure when it runs it via rlwrap)#2019-10-2519:08andy.fingerhutHuh, didn't know about -x option to bash. Looks generally useful for all bash scripts.#2019-10-2519:09seancorfieldYeah, I use it a lot for debugging scripts.#2019-10-2519:54alexmillerditto :)#2019-10-2520:22andy.fingerhutI mean, I knew about the set -x command by editing my own bash scripts to debug them, and sometimes did that on things owned by root if I had the permission, but nice that it can also be done without editing the script.#2019-10-2523:02sogaiuis there something comparable for powershell?#2019-10-2523:13sogaiuah, may be this: https://ss64.com/ps/set-psdebug.html#2019-10-2905:07johanHello, did anyone successfully use nRrpl and repl-y together to be launched via a clojure -A:repl ?#2019-10-2905:33seancorfield@johan178 Something like this? clj -Sdeps '{:deps {reply {:mvn/version "RELEASE"}}}' -e "((requiring-resolve,'nrepl.server/start-server))" -m reply.main
That will start an nREPL server on a random port (and report it) as well as starting REPL-y.#2019-10-2905:35seancorfieldThen you can connect into that server from other clients. I just tried connecting in with Leiningen to check it worked.#2019-10-2905:35seancorfieldYou could provide a specific port number if you wanted in the call to start-server.#2019-10-2906:36johan:nrepl {:extra-deps {nrepl {:mvn/version "RELEASE"}
reply {:mvn/version "RELEASE"}
cider/cider-nrepl {:mvn/version "RELEASE"}}
:main-opts ["-e" "((requiring-resolve,'nrepl.server/start-server))"
"-m" "reply.main"] }
I got an alias working.
Trying to get nrepl midlerware (cider-nrepl) working#2019-10-2911:18borkdude@johan178 this is what my cider-nrepl alias looks like: https://github.com/borkdude/clj-kondo/blob/master/doc/dev.md#toolsdeps#2019-10-2911:19borkdude(fwiw I don't start a REPL in terminal, I just connect to it from my editor)#2019-10-3105:23maxpIs usage of "RELEASE" / "LATEST" in :mvn/version documented somewhere?#2019-10-3105:36seancorfield@maxp They are not really recommended.#2019-10-3105:38seancorfieldThey are not supported in Maven 3, I believe. They are only available in Maven 2 based tools.#2019-10-3105:39seancorfieldRELEASE means the most recent non-snapshot release, so it can be an alpha, for example, or a "gold" version. LATEST means the most recent -SNAPSHOT release which can change frequently.#2019-10-3105:41seancorfieldThat said, I use RELEASE for dev-only tooling, as a "convenience" for it always being up-to-date but with the caveat that it could of course break any day. I never use LATEST.#2019-10-3105:41seancorfieldDoes that help @maxp?#2019-10-3106:36maxp@seancorfield - yes. Thank you/#2019-10-3116:31drewverleeIts possible to specify multiple deps files and have later ones override earlier ones right?#2019-10-3116:35alexmillerthere are multiple deps files that override, but that is not an open set#2019-10-3116:35alexmillerthe deps chain is: install deps.edn (now a resource in tools.deps.alpha), user deps.edn (~/.clojure/deps.edn), project deps.edn, -Sdeps#2019-10-3116:36alexmillerif you're using tools.deps directly, you can of course do whatever you like#2019-10-3116:36alexmillerit has tools for loading and merging deps.edn files#2019-10-3116:42drewverleeThanks alex.#2019-10-3116:58drewverleeso if you want to override a project dep it would be something like "-Sdeps {:aliases {:foo {:override-deps ...}}} -A:foo"#2019-10-3117:07alexmilleryou don't need an alias for that#2019-10-3117:07alexmillercan just -Sdeps '{:deps {the/lib {:mvn/version "whatever"}}}'#2019-10-3117:08alexmiller:deps merge in#2019-10-3117:08alexmilleryou can do what you said too though#2019-10-3117:12drewverleemakes sense.#2019-11-0314:19athomasoriginalI love cofield’s dot-clojure repo: https://github.com/seancorfield/dot-clojure/blob/master/deps.edn. I would like to make an alias in my own global deps.edn file that will use the latest version of ClojureScript e.g.
{:aliases
{:cljs
:extra-deps {org.clojure/clojurescript {:git/url ""
:sha "6431e6435ad9fc85e766752f4d6939ff978db5a5"}}
:main-opts ["--main" "cljs.main" "--repl-env" "node" "--repl"]}}}
But when I call the above, I get Error building classpath. Destination path "clojurescript" already exists and is not an empty directory. What am I missing here?#2019-11-0316:29athomasoriginalI did a little digging and was able to resolve my issue by executing the following steps:
1. Delete ~/.gitlibs/_repos/github.com/clojure/clojurescript (which was empty)
2. Run clj -A:cljs again
Now everything seems to be working again. Is there a command in the tools.deps API that can do this automatically?#2019-11-0316:30alexmillerI’ve seen a few people hit this but I have not been able to reproduce it and don’t have a ready fix for it#2019-11-0316:42athomasoriginalFair. I tried to setup a minimal repro based on what my previous dir structure looked like, but can’t get it right. The best I found is this other interesting scenario
1. Successfully run clj -A:cljs as I have in the code snippet above (then stop the repl)
2. delete content of ~/.gitlibs/libs/org.clojure/clojurescript + ~/.gitlibs/_repos/github.com/clojure/clojurescript (both clojurescript still exist, but they are empty)
Now when I run clj -A:cljs I get the following error
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate cljs/main__init.class, cljs/main.clj or cljs/main.cljc on classpath.
Full report at:
/var/folders/qc/mv_jl6857850zkr0qv6gj89m0000gn/T/clojure-5520844173070466309.edn
The above is resolved by deleting .cpcache#2019-11-0316:44athomasoriginalThe above seems odd, but that seem to be how my local looked at the time.#2019-11-0318:36borkdudefwiw I had a similar problem recently (went away after deleting .gitlibs and/or .cpcache)#2019-11-0320:55seancorfield@tkjone Or perhaps putting -Sforce on that clj command, to force it to rebuild the cache?#2019-11-0320:59sogaiui've been doing -Sforce more often than deleting .cpcache with some success recently -- iiuc this will leave older bits in .cpcache if you want to investigate issues later, helpful when reporting issues perhaps?#2019-11-0321:03seancorfieldYeah, you get a separate set of cached files for each alias combination you use, so it's nice to keep the ones that work fine and just override the ones that don't, via -Sforce.#2019-11-0411:05vlaaadAnother use-case for https://clojure.atlassian.net/browse/TDEPS-116 — I want to create a library that has different code for different jdk versions. I do it using 2 aliases:
{:jdk8 {:extra-paths ["src/jdk8"]}
:jdk11 {:extra-paths ["src/jdk11"]}}
It would be much easier to depend on such library when you can specify extra alias when declaring a dependency to it#2019-11-0411:45vlaaadI think I found a bug: :paths in aliases are not picked up when there are also :deps present. Repro:
deps.edn:
{:paths ["src" "jdk11"]
:aliases {:jdk8 {:paths ["src" "jdk8"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}}}}}
clj -Srepro -Sforce -Spath output:
src:jdk11:/home/vlaaad/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/home/vlaaad/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/vlaaad/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
clj -A:jdk8 -Srepro -Sforce -Spath output:
src:/home/vlaaad/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/home/vlaaad/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/vlaaad/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
notice: second output does not have jdk8 folder on a classpath#2019-11-0411:50vlaaad#2019-11-0411:50vlaaad#2019-11-0413:16stijn@vlaaad you have a typo in the deps.edn, :deps is not inside the :jdk8 alias.#2019-11-0413:17vlaaadwhat, where? it is!#2019-11-0413:18vlaaadoh, you are right#2019-11-0412:11dominicmIn maven that's a classifier I think. Although jdk11 had a system for having jdk-specific files added.#2019-11-0412:22dominicm@vlaaad jep238: multi release jars#2019-11-0412:23dominicmIt would be cool if deps.edn supported MRJAR for fetching dependencies.#2019-11-0413:01alexmillerI don’t think mrjars have any effect on deps? That’s really a JVM thing?#2019-11-0413:01alexmillerOn the problem above, what version of clj are you on?#2019-11-0413:06vlaaadlatest, 1.10.1.148#2019-11-0413:10vlaaad@dominicm will jep 238 work with clj files as well? is it okay to make library muti-release?..#2019-11-0413:10dominicmI see no reason why not. Try it and report back!#2019-11-0413:12vlaaadI think jep238 might not be a fitting solution for me, because I also need different dependencies for different java versions...#2019-11-0413:29alexmillerYeah, won’t do that, but should work for clj files as they are loaded as resources, just like class files#2019-11-0413:32vlaaadyeah, makes sense#2019-11-0413:34alexmillerThis is really where maven uses classifiers and jdk properties#2019-11-0413:10dominicmI was thinking of fetching the appropriate pom.xml from the jar based on jdk version#2019-11-0413:32alexmillerI don’t think that’s a thing#2019-11-0413:32alexmillerAlthough I have not looked at all at the maven support for mrjars#2019-11-0413:35dominicmIt doesn't look like it is. It could be for local/root on a jar, but for maven a separate fetch is made for the pom, completely unaware of the jdk#2019-11-0413:39vlaaadis there a way to exclude dependency using alias?#2019-11-0413:41alexmillerNot right now#2019-11-0413:54vlaaadsad, I'm running out of ideas how to add jdk8 support without breaking git dependencies to cljfx 😞#2019-11-0413:56vlaaadmaven is easy, I can build poms with different deps and source paths and publish them either as different artifacts, or with different classifiers/activations#2019-11-0413:57vlaaadbut I already have jdk11-specific deps in root deps.edn, and it seems there is no way to exclude them when using git dependency?#2019-11-0413:59alexmilleryou could try using a lib with nil coordinate - that will merge over, but I don't remember how tools.deps handles it (could easily not work, I just don't remember)#2019-11-0414:01vlaaadusing :override-deps?#2019-11-0414:01alexmilleryeah#2019-11-0414:04vlaaaddoesn't seem to work:
given this deps.edn:
{:paths ["src" "jdk11"]
:deps {org.openjfx/javafx-base {:mvn/version "13"}}
:aliases {:jdk8 {:paths ["src" "jdk8"]
:override-deps {org.openjfx/javafx-base nil}}}}
running clj -A:jdk8 -Srepro -Sforce -Spath still includes javafx-base:
src:jdk8:/home/vlaaad/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/home/vlaaad/.m2/repository/org/openjfx/javafx-base/13/javafx-base-13.jar:/home/vlaaad/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/vlaaad/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/home/vlaaad/.m2/repository/org/openjfx/javafx-base/13/javafx-base-13-linux.jar
#2019-11-0414:05vlaaadand using :deps in alias seems to interfere with :paths so jdk8 path is not seen, as reported earlier#2019-11-0414:05alexmilleryeah, I'm going to look at that in a bit once I'm past my morning calls#2019-11-0414:06alexmillerthat's surprising to me#2019-11-0414:07alexmillercould you also post me your clj -Sdescribe and whether you have anything in ~/.clojure/deps.edn?#2019-11-0414:09alexmillerwait, you said latest but 1.10.1.148 is not a thing, should be 1.10.1.478#2019-11-0414:10vlaaadI'm using -Srepro, so ~/.clojure/deps.edn shouldn't be an issue?#2019-11-0414:10vlaaadI'm using 1.10.1.478, it was a typo#2019-11-0414:10alexmillerah, gotcha#2019-11-0414:10alexmilleron both#2019-11-0414:11vlaaadclj -A:jdk8 -Srepro -Sforce -Sdescribe
{:version "1.10.1.478"
:config-files ["/usr/local/lib/clojure/deps.edn" "deps.edn" ]
:config-user ""
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/vlaaad/.clojure"
:cache-dir ".cpcache"
:force true
:repro true
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ":jdk8"}
#2019-11-0414:16alexmilleroh, this is the expected behavior#2019-11-0414:16alexmillerthe :paths and :deps in an alias are new with the newest release and REPLACE the project :paths and :deps#2019-11-0414:17alexmillerif you want to add, you need :extra-paths or :extra-deps#2019-11-0414:17vlaaadI want to replace both#2019-11-0414:17alexmillerI thought your complaint above was that the project :paths was not included?#2019-11-0414:18alexmillerre-reading what you said above...#2019-11-0414:18vlaaadNo, replacement paths are not picked up#2019-11-0414:51alexmillerah, it is a bug and I see the fix#2019-11-0415:14alexmillerand there is a test that should have caught this, but the test is missing the is, so it didn't fail :)#2019-11-0415:14alexmillerprogramming is hard, y'all#2019-11-0415:31andy.fingerhuteastwood will probably warn about such tests, if it works on your code base at all.#2019-11-0416:03kszaboclj-kondo as well#2019-11-0415:46alexmiller@vlaad logged as https://clojure.atlassian.net/browse/TDEPS-142, fixed, and released in tools.deps.alpha 0.8.584, the rest is working its way through to clj#2019-11-0416:09vlaaadthanks a lot!#2019-11-0416:11alexmillerthanks for a concise and useful report! :)#2019-11-0416:07alexmillerI didn't that see that with clj-kondo, but I'm not running latest version#2019-11-0416:08borkdudedepends on the example. what's the code?#2019-11-0416:10borkdudefound it. yes, that's a case I want to improve. currently it only checks the direct children of (deftest ...) like (deftest foo (= 1 2)), but these cases were inside a let: https://github.com/borkdude/clj-kondo/issues/462#2019-11-0416:30andy.fingerhutThere are also typically many is/are expressions inside of testing subexpressions inside of deftest, or let/testing/let/... etc.#2019-11-0416:31borkdudeyeah. does eastwood test for that btw?#2019-11-0416:34andy.fingerhutEastwood uses tools.analyzer to analyze everything, then looks for all expansions of is forms, I believe.#2019-11-0416:35andy.fingerhutIt is one way to do it, but can be painful to recognize the expansions of is, which is an odd macro.#2019-11-0416:35borkdudebut the point is to detect that is was forgotten. so it warns when you have a deftest without a single occurrence of is?#2019-11-0416:35borkdudeglobally? or does it also point to the location where the is was expected?#2019-11-0416:36andy.fingerhutI'd have to go back to the code to check, but I think it warns about all expressions that are not likely to be called only for their side effects, that do not have is or are around them.#2019-11-0416:37andy.fingerhutI'll follow up in #eastwood channel after checking the code for a little bit.#2019-11-0417:10kszabo@alexmiller If you run the tests through Kaocha, then this might have caught it as well: https://github.com/lambdaisland/kaocha/blob/53d06ab7c56e48cec19bc2ab41a98e0f7b2b860c/doc/clojure_test/assertions.md#clojuretest-assertion-extensions#2019-11-0417:10kszabo@alexmiller If you run the tests through Kaocha, then this might have caught it as well: https://github.com/lambdaisland/kaocha/blob/53d06ab7c56e48cec19bc2ab41a98e0f7b2b860c/doc/clojure_test/assertions.md#clojuretest-assertion-extensions#2019-11-0417:12borkdudeI guess even clojure.test could have built-in warnings like: you're defining a deftest but you never called is.
Or: you called testing, but you never called is.#2019-11-0417:50alexmillerBut it’s difficult to verify that beyond lexical scope#2019-11-0417:54borkdudeI'm not sure how clojure.test works internally, but is there not some kind of registry to which test an assertion belongs?#2019-11-0417:56borkdudeThere are all kinds of hooks. Maybe it's possible to verify during one of those hooks that nothing got actually checked when running a test.#2019-11-0417:57alexmillerPossibly #2019-11-0417:57alexmillerI’ve also written tests with negative assertions in code that shouldn’t run in the past. In those cases, even no assertions is correct#2019-11-0417:58alexmillerBut those are prob rare#2019-11-0418:02borkdudethat's meta: assert on a hook that the number of assertions is zero#2019-11-0418:07alexmilleryou could also write those kind of tests as a zero? check on a "reached" count in an atom I suppose so there would always be an assertion#2019-11-0418:08borkduderight#2019-11-0418:10alexmiller@U08E8UGF7 the build here is a Maven build (as with all contrib builds) so not trivial to integrate#2019-11-0418:10alexmillerironically of course, given it's tools.deps#2019-11-0418:13borkdudeit seems the feature we discussed for clojure.test is something kaocha also does, checking at runtime when the test ends using a hook:
https://github.com/lambdaisland/kaocha/blob/c97a2cbfbb029604edd42e1340c66bd4704007cf/src/kaocha/type/var.clj#L13#2019-11-0418:18borkdudeI would like to see this in clojure.test, but I think the chances are fairly slim it would ever get in#2019-11-0418:18borkdudebut as a library it could work#2019-11-0418:21kszabothis is one of the many reasons why we use kaocha, of course a static analysis check would be better as well. The negative assert feature could be fixed with dedicating another name for it like (clojure.test/never-reached!) which macros could check for. If I got the negative assert idea correct#2019-11-0418:37borkdude@alexmiller https://gist.github.com/borkdude/c43d5b8548d9185f8454bcced3861064#2019-11-0419:01borkdudehttps://github.com/borkdude/naw#2019-11-0419:23kszabocool! so productive, @U04V15CAJ 🙂 Can you pretty please add a deps.edn as well for us tools.deps users.#2019-11-0419:24borkdudeon it 🙂#2019-11-0419:28borkdude@U08E8UGF7 it's in there now. I'm trying to make it work under CLJS as well, but registering the multimethod doesn't seem to work#2019-11-0420:50borkdudeNow also works in ClojureScript.#2019-11-0422:29borkdudeFound an error in one of my own projects!
https://twitter.com/borkdude/status/1191482400599158785#2019-11-0504:02alexmillerif anyone has clj on windows and can check https://ask.clojure.org/index.php/8820/why-does-not-merge-deps-across-multiple-aliases-in-tools-deps, that would be cool. I tried it with this deps.edn on mac and couldn't repro:
{:aliases
{:a {:extra-deps {org.clojure/data.xml {:mvn/version "0.2.0-alpha6"}}}
:b {:extra-deps {org.clojure/data.json {:mvn/version "0.2.6"}}}}}#2019-11-0504:05seancorfieldTaking a look...#2019-11-0504:09seancorfieldThere's definitely something odd going on: PS C:\var\www\async.http\sketch> clojure -R:a -R:b -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.xml\0.2.0-alpha6\data.xml-0.2.0-alpha6.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;C:\Users\seanc\.m2\repository\org\clojure\data.codec\0.1.0\data.codec-0.1.0.jar
PS C:\var\www\async.http\sketch> clojure -A:a -A:b -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.xml\0.2.0-alpha6\data.xml-0.2.0-alpha6.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;C:\Users\seanc\.m2\repository\org\clojure\data.codec\0.1.0\data.codec-0.1.0.jar
PS C:\var\www\async.http\sketch> clojure -Sforce -A:a -A:b -Spath
WARNING: Specified aliases are undeclared: [:a ]
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar
PS C:\var\www\async.http\sketch> clojure -Sforce -R:a -R:b -Spath
WARNING: Specified aliases are undeclared: [:a ]
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar
PS C:\var\www\async.http\sketch>
#2019-11-0504:10seancorfieldI got the same results from -R and -A tho' in all cases (including -A:a:b and -R:a:b, not shown) but with -Sforce, the first alias is somehow not found...#2019-11-0504:12seancorfieldAfter running that -Sforce variant, running the command without -Sforce is now behaving as if :a was not included.#2019-11-0504:12alexmillerthat extra space in [:a ] is probably important - I presume that's causing the alias not to be found there#2019-11-0504:13seancorfieldPS C:\var\www\async.http\sketch> clojure -R:a:b -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar
PS C:\var\www\async.http\sketch> clojure -R:b:a -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\data.xml\0.2.0-alpha6\data.xml-0.2.0-alpha6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;C:\Users\seanc\.m2\repository\org\clojure\data.codec\0.1.0\data.codec-0.1.0.jar
Note that swapping the aliases produces the correct result (since it's treated as a new invocation)#2019-11-0504:15seancorfieldThen I ran it with -Sforce -R:a:b and got the correct behavior back. So I'm suspecting a problem with parsing options if -Sforce is used?#2019-11-0504:18seancorfieldYeah, almost certainly something about -Sforce messing up alias parsing or combining somehow based on further experiments.#2019-11-0504:20seancorfieldThat's with PS 1.10.1.469 BTW.#2019-11-0504:22seancorfieldAnd it's peculiar to the PS script. The exact same sequences of commands on Linux just work.#2019-11-0504:25seancorfield(how did he get .483 on Windows? I tried scoop update clojure and it thinks .469 is the latest... maybe the wiki page has not been updated?)#2019-11-0504:25alexmillerI updated the wiki page#2019-11-0504:25alexmillerassuming you mean https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#2019-11-0504:26alexmillerI don't anything about the scoop thing though#2019-11-0504:26seancorfieldYeah, the scoop script is supposed to use that link in that page to pick up a newer version...#2019-11-0504:26alexmillerI wouldn't expect these to differ in this though#2019-11-0504:27alexmillerso it's likely same in 483#2019-11-0504:27seancorfieldFair enough. Now I'm still puzzled about the scoop update process 🙂 I'll open an issue...#2019-11-0504:27alexmillerI assume these are bugs in the ps script#2019-11-0504:42seancorfieldInteresting. Now I've managed to get this in a clean folder without using -Sforce: PS C:\var\www\async.http\sketch> clojure -A:a -A:b
WARNING: Specified aliases are undeclared: [:a ]
Clojure 1.10.1
user=>
#2019-11-0504:43seancorfieldThat makes a bit more sense given the report on ask...#2019-11-0504:46alexmiller[:a ] has an extra space which I assume is the problem#2019-11-0504:47seancorfieldI think all of these need to be wrapped in $(,,, -join '') https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/ClojureTools.psm1#L259-L273#2019-11-0504:47seancorfieldIn the same way as is done here https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/ClojureTools.psm1#L224#2019-11-0504:48seancorfieldI suspect the default conversion for multi-element collections is to have spaces between the elements.#2019-11-0504:48alexmillerthe tools.deps code splits on colons, so could easily end up with keywords with spaces in them#2019-11-0504:48seancorfieldSo $ToolsArgs += "-R$($ResolveAliases -join '')"
etc#2019-11-0504:51alexmillerboth $ there?#2019-11-0504:51alexmillerguess so#2019-11-0504:53seancorfieldYup, that's the fix. I just updated my .psm1 script to have that and it works as expected now.#2019-11-0504:53seancorfieldAll of those $ToolsArgs += lines need that...#2019-11-0504:54alexmillerhttps://github.com/clojure/brew-install/commit/cc4fba82aee7e8a83269eed666dcfd1180e5069a ?#2019-11-0504:55seancorfieldLooks good!#2019-11-0504:56alexmillerok, there's a windows build out there at .486 now#2019-11-0504:57alexmillerhttps://download.clojure.org/install/win-install-1.10.1.486.ps1#2019-11-0504:57seancorfieldWiki page updated?#2019-11-0505:00alexmilleryes#2019-11-0505:02alexmillerheading to bed, thx for the help#2019-11-0505:37andy.fingerhutI tried installing Homebrew then brew install clojure on a couple of Ubuntu Linux VMs with different versions, and they both installed version 1.10.1.478 still. brew update clojure on my Mac upgraded to 1.10.1.483, so perhaps Homebrew Linux is taking longer to update package info somewhere?#2019-11-0703:23seancorfield@U0CMVHBL2 FWIW, I installed brew on Ubuntu (actually WSL on Windows 10), and then brew install clojure installed 1.10.1.483.#2019-11-0703:32andy.fingerhutCool. I will check on an Ubuntu Linux system again soon and report back, to see if it gets the latest version now.#2019-11-0505:39andy.fingerhutStrangely enough, though, brew install clojure worked fine on those Linux machines, with no problem finding a working version of ruby executable to do what it needed to install the clojure thingy (replace "thingy" with correct Homebrew term, whatever that is)#2019-11-0512:45alexmillerFormula#2019-11-0512:45alexmillerMaybe Linux brew is downstream somehow#2019-11-0521:53seancorfieldAnother Windows glitch? Can someone else on Windows PowerShell confirm/deny this for me? PS C:\Users\sean> clojure -Sdescribe
{:version "1.10.1.486"
:config-files ["C:\Users\sean\scoop\modules\ClojureTools\deps.edn" "C:\Users\sean\.clojure\deps.edn" "deps.edn"]
:config-user "C:\Users\sean\.clojure\deps.edn"
:config-project "deps.edn" when run in a folder that does not have it still reports it in describe and as the :config-project path. The macOS/Linux version displays it as :config-project (which still seems wrong) but correctly omits it from the :config-files path.#2019-11-0521:57alexmillerThe behavior in Mac is right - it’s telling you where it’s looking#2019-11-0521:57alexmillerconfig-files is not actually used now#2019-11-0521:58alexmillerThe Clojure code will do the right thing if it’s missing#2019-11-0522:02seancorfieldOK, so a small glitch in the PS script but not an important one.#2019-11-0522:03seancorfieldmacOS (! 720)-> clojure -Sdescribe
{:version "1.10.1.483"
:config-files ["/usr/local/Cellar/clojure/1.10.1.483/deps.edn" "/Users/sean/.clojure/deps.edn" ]
:config-user "/Users/sean/.clojure/deps.edn"
:config-project "deps.edn"
#2019-11-0522:03alexmillerSeems ok to me#2019-11-0522:07seancorfieldGiven that tooling should no longer be running clojure -Sdescribe via (clojure-env), and such tooling was broken on Windows anyway(!), I agree that nothing really needs to be done. It's just slightly odd to look at.#2019-11-0522:09alexmillerconfig-files is not even passed to the script anymore#2019-11-0614:14Michaël SalihiHello everybody, I would to know if I can kick off a one command line simple http server with Clojure like Python does python3 -m http.server ?
With tools.deps for eg. ? clj -Sdeps '{:deps {some/clojars {:mvn/version "x.x.x"}}}' -m http.server#2019-11-0615:40souenzzohttps://github.com/kachayev/nasus#usage#2019-11-0617:30Michaël SalihiPerfect ! Thx @U2J4FRT2T#2019-11-0615:27dominicmI think there was a snippet for that kicking around#2019-11-0617:26rickmoynihan@dominicm: I’m currently using the capsule builder for pack.alpha, but running into annoyances it brings wrt the JVM trampoline, tunnelling args and also wanting to extend the classpath of the “uber.jar” from the outside via java -cp uber.jar:./my-file ,,,. Is the one-jar builder likely to cause me less friction?#2019-11-0617:27rickmoynihantheir docs describe the problem, but not really their solution other than as a custom classloader that redirects requests.#2019-11-0618:04dominicmHaven't used either in this way.#2019-11-0618:04dominicmOneJAR will likely work well for this case, based on my recollection of its function. It'll delegate.#2019-11-0703:24seancorfieldHow to confuse yourself: in Ubuntu bash on Windows (via WSL 1), type scoop update clojure, and wonder why, after completing successfully, when you type clojure -Sdescribe it shows the version you previously had installed...#2019-11-0703:25seancorfield...then switch to PowerShell and run clojure -Sdescribe and realize that it was updated: Windows figured out scoop was a Windows command, even tho' you typed it into Ubuntu, and ran it on Windows anyway... 👀#2019-11-0703:33andy.fingerhutI'm sticking with Linux in a VM, if I ever use a Windows machine regularly 🙂#2019-11-0703:34seancorfieldWSL 2 is a full VM-based approach. I'm stuck with WSL 1 for... reasons... on both of my Windows systems.#2019-11-0703:34seancorfield(I run Windows 10 on my Mac, BTW, and pretty much only use macOS for Terminal and Docker!)#2019-11-0704:09drewverleeso the command here: clj -A:pack mach.pack.alpha.jib --image-name --image-type registry -m my.main
from https://github.com/juxt/pack.alpha. seems to suggest that by passing -m its going to record that as the main namespace. but whats happening is that its "running" that namespace. which makes sense given that passing -m to clj does just that.
what i'm i missing?#2019-11-0704:10drewverleehmmm#2019-11-0704:12drewverleeactually that might not be the issue. is mach.pack.alpha.jib a script/function im passing clj and the rest are the args?#2019-11-0704:12drewverleeyea, thats probably right#2019-11-0704:13drewverleemy issue was that i told the pack alias to run the main function.#2019-11-0707:41andy.fingerhutI tried on an Ubuntu 16.04 Linux machine (freshly installed, latest updates from Canonical as of today), to install Homebrew for Linux, then brew install clojure. It installed the latest version 1.10.1.483 which brought up a REPL just fine. The Linux VM I tested on had no previous installation of ruby, clojure, nor Java. brew install clojure installed them all automatically as dependencies, choosing AdoptOpenJDK 8 for the JDK (plus some things that surprised me, but didn't hurt -- e.g. gcc seemed weird)#2019-11-0710:56robert-stuttafordProbably a very-FAQ, but googling didn't help me at all. how do teams deal with sharing deps.edn files that use :local/root? one option is to symlink the common code folder on everyone's machines, but i'm wondering if the lib itself supports something out of the box#2019-11-0710:58dominicmIt depends 🙂#2019-11-0710:58dominicmIf they're released separately, then I wouldn't use :local/root#2019-11-0710:59robert-stuttafordthis is just to support a short period of active new development#2019-11-0710:59robert-stuttafordrelease will use the usual s3 maven repo nonsense, until we can get everything transitioned to deps, and then we'll use the git sha thing#2019-11-0711:04robert-stuttafordis there inbuilt capability to resolve local relative paths, or is symlinks the way?#2019-11-0711:06dominicmjust as a workflow thing for what I do: as someone doing multi-repo development I personally setup aliases in my ~/.clojure/deps.edn file.#2019-11-0711:06dominicmSome of my colleagues don't do any cross-development, so they don't have to care. When I do the cross development, I just activate the extra profile(s) on launch.#2019-11-0711:06dominicmrelative paths do work though.#2019-11-0711:08robert-stuttafordah, i tried relative to home ~/Code/... but not relative to folder ../..., going to try that now#2019-11-0711:09robert-stuttafordthanks Dominic!#2019-11-0711:09dominicmYeah, no support for ~. I think it's been discussed & rejected. .. works though, used heavily in Edge.#2019-11-0711:16robert-stuttafordnice!#2019-11-0722:00drewverleeif your projects deps.edn uses git/url to point at another project, does it have to also have a deps.edn? or would a project.clj work? i dont think so, but its been a while.#2019-11-0722:01kszabothe former#2019-11-0722:01drewverleesorry, could you clarify? i asked my question awkwardly. It must have a deps?#2019-11-0722:03alexmillerit must have either a deps.edn or a pom.xml#2019-11-0722:03alexmilleror neither#2019-11-0722:04alexmillerdeps does not currently support project.clj as a manifest as there is no good way to canonically load it and find the deps without basically running all of lein#2019-11-0722:04drewverleeright, thanks alex.#2019-11-0722:04alexmillerbecause project.clj is evaluated, can be modified by plugins, etc#2019-11-0808:47plexus@drewverlee maybe obvious but worth noting, if you control the repo you can just lein pom and check in the pom.xml next to the project.clj#2019-11-0811:07vlaaadI'm starting to feel that both lein pom and clj -Spom is a case of easy over simple. There is a dependency format — pom, that has many different stuff in it. It's much more than list of deps + artifact + version, but tools like lein pom and clj -Spom give only the most common baseline, which might cover some percentage of use cases, but whenever I want to publish a library, I realize that that's not enough. I need to specify developer/license information. I need to specify version as tag in scm information. Someone else will want to specify git sha instead of tag in scm. Dependencies I use during development might go to different profiles, with different activation rules, so what lein/`clj` provides to update dependency list might even be wrong.
I guess ideally I want to produce pom from code I write, aided with information from tools-deps/git and some helpers to generate xml from data, but ultimately be the one who decides what goes where. I guess where is no such tool yet because you can just check in pom.xml and update it with scripts using sed or whatever, and that's good enough.#2019-11-0811:29dominicmUsing sed for xml was hard :) Garamond works okay in this space.#2019-11-0811:30dominicmI think it's okay to say that pom.xml is a combination of data sources into one. I personally would not want to manage the tag/version stuff as I'd definitely get it wrong.#2019-11-0815:37drewverleeInteresting. So assuming no plugin modifies the deps. then its as easy as generating a pom for say a project like: https://github.com/weavejester/integrant?#2019-11-0815:38dominicmYup#2019-11-0815:38dominicmAlthough a deps.edn is probably easier#2019-11-0815:43drewverleewhat would the output of clj -Spath | grep integrant containing integrant twice imply? Im trying to undertsand a build error.
/home/drewv/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src:/home/drewv/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src/main/clojure#2019-11-0815:45alexmillerit doesn't contain integrant twice#2019-11-0815:45alexmillerit contains two source directories from within integrant (src and src/main/clojure)#2019-11-0815:47alexmillerthe latter doesn't exist, so no worries#2019-11-0815:47alexmiller(that's the standard Maven source path for Clojure source files)#2019-11-0815:49drewverleethanks alex. So {:warning "could not find classpath entry", :path "/home/drewv/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src/main/clojure"} isn't indicating something is wrong with my build? its because ____ produces that path and then a later tool can't find the entry on it?#2019-11-0815:52alexmillerI don't know what that is#2019-11-0815:52alexmillerwhere do you see it?#2019-11-0815:52alexmillerit's certainly not an issue to the jvm to have a classpath entry that doesn't exist#2019-11-0815:53alexmillerwhich I assume is why that's a warning#2019-11-0815:54drewverleethats the output from running depstar https://clojars.org/seancorfield/depstar.
Right, makes sense.
thanks#2019-11-0819:24seancorfield@drewverlee Integrant doesn't have a deps.edn file and that path indicates you are depending on it via a :git/url?#2019-11-0819:25drewverleeIt's a fork.#2019-11-0819:27seancorfieldAh, with a pom.xml file. Got it.#2019-11-0819:28seancorfield@alexmiller Does t.d.a. assume src/main/clojure when it sees a POM file in a Git dep?#2019-11-0819:29lreadhello! I am wondering, should :override-deps affect the pom.xml generated/updated by -Spom?#2019-11-0819:29seancorfield@lee If you specify an alias that brings in that override...#2019-11-0819:30seancorfield(at least, that's why I would expect -- I've never tried it)#2019-11-0819:32lreadYep, I am specifying the alias and it does not affect pom.xml in my little test. Could be I botched something.#2019-11-0819:32seancorfieldBased on a quick test, it doesn't seem to.#2019-11-0819:33seancorfield(! 815)-> cat deps.edn
{:paths ["resources" "src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}}
:aliases
{:test {:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}}}
:old {:override-deps {org.clojure/clojure {:mvn/version "1.9.0"}}}
:runner
{:extra-deps {com.cognitect/test-runner
{:git/url ""
:sha "76568540e7f40268ad2b646110f237a60295fa3c"}}
:main-opts ["-m" "cognitect.test-runner"
"-d" "test"]}}}
Fri Nov 08 11:33:00
(sean)-(jobs:0)-(~/clojure/pommy.example)
(! 816)-> clj -A:old
Clojure 1.9.0
user=> ^D
#2019-11-0819:33seancorfieldBut clj -A:old -Spom produces a pom.xml that uses 1.10.1 not 1.9.0#2019-11-0819:35lreadthanks sean! -Stree behaves as expected:
> clj -A:old -Stree
org.clojure/clojure 1.9.0
org.clojure/spec.alpha 0.1.143
org.clojure/core.specs.alpha 0.1.24
#2019-11-0819:38seancorfieldI'd say that's a bug. @alexmiller?#2019-11-0820:26alexmillerthis is trickier than it looks. I may actually have broken it in my recent set of changes. the tricky part is that the aliases are about resolving transitive deps and building a classpath, but you're not actually building a classpath here, you're just making a different manifest file#2019-11-0820:28alexmillerso if you supplied override-deps for some lib N levels down, what does that mean? do you add it to the root deps to get the same effect? or just ignore it?#2019-11-0820:54seancorfieldI would certainly expect the dependencies in pom.xml to match the top-level deps in -Stree output but I agree that it's really hard to decide what the transitive dependencies should be. Hmm 🤔#2019-11-0821:03alexmillerI used to do a bunch of fiddly logic to half apply stuff from the aliases during gen, but I stopped doing that in my last round of changes and just reused the same core from making classpaths, but did not properly take that all the way#2019-11-0821:38lreadForgive my naive question @alexmiller, I'm sure you've put much thought into this. Wouldn't adding the override of the transitive dep to the root deps give the intended result?#2019-11-0821:39lreadAlso do you want me to raise an issue for this? Or is it via ask clojure now?#2019-11-0821:50alexmillerI've actually fixed it locally, so no need to do anything. I'll probably log a jira just for tracking before I commit it#2019-11-0821:51alexmilleron the transitive deps, the decision I had made in the past was to only modify the root deps and not include overrides in transitive deps#2019-11-0821:52alexmillerand that's what I'm matching in the update. inevitably, all decisions here will be wrong for someone.#2019-11-0821:55lreadtrue enough! So long as behavior is consistent and easy to understand, it works for me! (for what that's worth!)#2019-11-0822:16alexmiller@lee https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-143, fix in master but not released#2019-11-0822:16lreadcool, thanks alex!#2019-11-0921:52dominicmhttps://github.com/stuarthalloway/lancet perhaps interesting to this group 🙂#2019-11-0922:05sogaiusomehow i am reminded of boot and badigeon#2019-11-0922:06sogaiu@dominicm thanks, had never seen that 🙂#2019-11-0923:26alexmillerLancet was the long running example in the first edition of Programming Clojure#2019-11-1220:03alexmillerI'm going to do a talk about tools.deps / clj next week at the conj - any burning questions I should be sure to answer?#2019-11-1220:06dominicmSettle the debate on namespaces. Talk about java. Leak the plans for tools.ant#2019-11-1220:07seancorfield@alexmiller Talk about add-lib 🙂#2019-11-1220:10alexmiller@dominicm if those are serious, I'm not sure what any of them mean#2019-11-1220:12dominicmI'm just teasing :) I was hoping tools.ant would give it away.
Unless there really is a tools.ant...#2019-11-1220:13alexmillerwell, Lancet was kind of like tools.ant :)#2019-11-1220:16dominicmIt would be interesting to know if there's ideas you have that you'd like others to execute. (E.g. Expanding on Lancet, building X, the community would gladly step up and at the least explore the domain for you)#2019-11-1220:19alexmilleryeah, I'll cover some future idea stuff for sure#2019-11-1220:11alexmiller@seancorfield maybe, we'll see#2019-11-1220:11dominicmIn earnest, the big questions I've had are:
- how do I do java
- how do I do thing I do with lein (usually uberjar)
- how do I load the tests of my dependencies (usually in the context of a monorepo)
- why would I use this rather than lein#2019-11-1220:12alexmiller"do java" == "compile java"?#2019-11-1220:13dominicmSorry, yes. How do I fold java into my program, or use java libraries from source. Stuff like that.#2019-11-1220:13alexmillerdon't? :)#2019-11-1220:14dominicmGood slide, yep :) (Although the team I said that to then introduced lein and took away all my lovely deps.edn)#2019-11-1220:17seancorfieldNot so much "do java" but "do compile" in the presence of gen-class etc I think would be interesting to at least mention (`compile`, binding the output directory, making sure it exists).#2019-11-1220:18alexmilleryou guys are too far on one end of the audience distribution :)#2019-11-1220:18dominicmThese are really clojure questions, but is interesting how lein did all this stuff that we didn't understand. Even though it's simple to do yourself.#2019-11-1220:19alexmillerbut I will keep those in mind#2019-11-1220:20dominicmI guess the recurring theme is "lein did X, deps.edn doesn't, where do I go?"#2019-11-1220:21seancorfieldhttps://github.com/seancorfield/cfml-interop/blob/master/deps.edn -- does compile of a gen-class namespace (caveat about the compile target dir needs to exist). I don't think compile/AOT is that far too one end of the spectrum really -- lots of people seem to assume you need AOT because "everything" out there assumes :aot main.core and uberjar with a compiled -main 🙂 That's a big difference when you switch to CLI/`deps.edn` from the lein world (or even the boot world).#2019-11-1220:26seancorfieldIntro to lein tends to be lein new app myapp; lein run; lein test; lein uberjar; java -jar path/to/the.jar -- so I think the equivalent story with CLI/`deps.edn` needs to be out there "officially" with at least some explanation of the differences.#2019-11-1220:27seancorfield(the current docs expect you to create your need project completely manually, for example, right?)#2019-11-1220:36alexmillerwhere "build" == what?#2019-11-1220:44seancorfield@alexmiller edited. Sorry, meant "create".#2019-11-1220:36alexmillercreate and deploy an artifact?#2019-11-1220:37alexmiller"build" by itself is not a very precise word in this context#2019-11-1220:37vlaaadhow to build your own lein with tools-deps 🙂#2019-11-1220:38vlaaadwhen TDEPS-116 will be solved 😄#2019-11-1220:39vlaaad(link if you are interested: https://clojure.atlassian.net/browse/TDEPS-116)#2019-11-1220:39alexmillerit'd be a lot better if that ticket talked about a problem, not a solution#2019-11-1220:39vlaaadit is#2019-11-1220:40alexmillerwell, not by my reading#2019-11-1220:41vlaaad...just not very clearly: I want to author more fine-grained libraries that may have optional code/dependencies which won't bloat classpath when unneeded. With maven it can be done using classifiers. With gitlibs, it's all or nothing#2019-11-1220:42alexmillerI have like 3 questions about every part of that statement#2019-11-1220:43vlaaadI'm ready to answer!#2019-11-1220:43alexmillerbut I'd suggest pulling the "optional code/dependencies" string as a place to start#2019-11-1220:45vlaaadI don't have an account there unfortunately, can't edit the issue.#2019-11-1220:45alexmillerI don't have time to have a long conversation about this right now but something like that is a far better title/thing to describe in a ticket.#2019-11-1220:46alexmiller"Allow specifying aliases in coordinates that point to deps.edn projects" is a specific change that might solve a problem, but unlikely to be the only possible change to solve those problems#2019-11-1220:48alexmillerMaven classifiers by themselves are just another dimension to your artifact coordinate (and tdeps already supports classifiers). They only get at this kind of a problem in combination with the Maven property system that allows you to conditionalize a dependency coordinate.#2019-11-1221:12sogaiumay be it's not worth saying to this audience, but does the following seem off?
my general impression was that clj / deps.edn tries to address some key issues (e.g. pomegranate-related, git deps, etc.) and wasn't out to replace lein, boot, etc. i thought a somewhat natural path would be for existing tooling to start using clj / deps.edn (or tools.deps, gitlibs, etc.) to take advantage of its capabilities. iiuc, there are also new efforts to build on or make use of tools.deps + related: https://github.com/juxt/pack.alpha, https://github.com/EwenG/badigeon, etc.
i've used at least one lein plugin (https://github.com/reifyhealth/lein-git-down) which, iiuc, makes use of gitlibs with varying degrees of success -- and iiuc there are other plugins (see the bottom of lein-git-down's README for some) that have made attempts. may be that these or similar things haven't "made it into the core of lein" is where some pain comes from?#2019-11-1221:23andy.fingerhutIf your future work part drops any hints about the more recent version of codeq tht you mentioned a few weeks ago in chat, you might be inundated with dozens of requests for more info, unless it is provided during your talk 🙂#2019-11-1221:39alexmillerthat has nothing to do with tools.deps so dont see why it would come up#2019-11-1221:30seancorfield@sogaiu I went down that path with boot-tools-deps but ultimately it was too difficult to meld t.d.a. and Boot due to their different worldviews on paths and dependencies for anything but the very simplest use cases. I don't know how lein-tools-deps fares in that regard.#2019-11-1221:32seancorfieldI think if lein/`boot` adopted t.d.a. wholesale as their way to handle paths and dependencies at their core, then that "natural path" could happen -- but that's a lot of rework and potentially breaking as well.#2019-11-1221:35seancorfieldAnd it's turned out that the effort that might have gone into that has perhaps gone into building tools on top of t.d.a. that provide the Leiningen-like features as point solutions you can mix'n'match (which is, to some extent, Clojure's general mindset). Both Leiningen and Boot are "easy" rather than "simple" because they are both "batteries included" -- which have served the community well for a long time but the CLI and the t.d.a.-based tools we have now feel more "Clojure-y" to me.#2019-11-1221:41sogaiuthanks for the insights#2019-11-1221:45vlaaadall "burning questions" for Alex to answer seem to be answered immetiately 🙂#2019-11-1221:48vlaaadI agree wholeheartedly with Sean, tools-deps are simple and that empowers me to quickly hack together tools I personally need, because I don't have to deal with semantics imposed by build tool#2019-11-1221:50vlaaadI think there is a lot of space for libraries to emerge that help with that process of developing projects#2019-11-1221:54vlaaadI made a couple of libraries with tools-deps, and the thing that wasn't there is aid to generate pom.xml, for example. clj -Spom is not enough — it's a good starting point, but my build scripts usually have to update different fields in pom, and dependency list provided by tools-deps might end up in different place... I'd like to see a tool that just helps me generating pom from data, and then I will use tools-deps programmatic APIs to provide information I need#2019-11-1221:58seancorfieldYeah, having a minimal pom.xml that satisfies the needs of clojars/cljdoc would be a big improvement over what -Spom produces by default but that would be a fairly simple tool to write I guess. Also something to update the version/tag in the POM file.#2019-11-1221:59seancorfield(shouldn't be core t.d.a./CLI functionality -- the community can easily build this... I just haven't quite felt enough pain yet to do write it myself 🙂 )#2019-11-1221:59kszaboit seems metav just added POM generation with versioning just yesterday 🙂
https://github.com/jgrodziski/metav/commit/e0bfe47f3af0659ac2a0bc624db82f90e2e0a6fe#2019-11-1222:00seancorfieldI could probably even build that into clj-new to be honest so at least you start with a somewhat fleshed out pom.xml file...?#2019-11-1222:00kszabohttps://github.com/jgrodziski/metav/issues/11#2019-11-1222:01kszabo> Pom.xml generation
>I have added the option to generate a pom.xml file based on what is already available from clojure.tools.deps. It actually uses the sync-pom function from tools deps then goes to the generated/updated pom and adds the group-id, artefact-name and version based on what metav already establishes.#2019-11-1222:02kszabosource: https://github.com/jgrodziski/metav/pull/12#2019-11-1222:01kszabothis looks like it fits the bill#2019-11-1222:04kszabowe are very happy users of metav so this is a welcome addition, maybe it would be nice to add this dependency to clj-new? We are mainly using it’s sha-based versioning but of course folks could use it for semver (who haven’t watched Spec-ulation 🙂 )#2019-11-1222:06alexmillerI haven't looked at it in many years, but Maven actually has support for storing pom data in arbitrary formats and there was even a clojure format way back when this was conceived of#2019-11-1222:08alexmillermaybe that's just taking on a lot more problems :)#2019-11-1222:10dominicmIs there a dimension in which maven is not extensible?#2019-11-1222:10alexmillersadly, no#2019-11-1222:13alexmillerhttps://stackoverflow.com/questions/11170057/what-happened-to-maven-polyglot#2019-11-1318:42rickmoynihanHow do people manage and maintain their aliases?
For example in one of our projects:
To start a basic dev repl (with all configured customers data on the classpath) you need the aliases: -A:dev:test:cljs:prj/all
For me to start a dev repl I need/want to add the extra aliases :cider:sc to those.
To run the tests in CI or via kaocha you need: -A:kaocha:test:prj/all:cljs
To build a specific release for a specific customer we need: -A:prj/customer-1:prod
Plus there are various other configurations of aliases. Right now I tend to “document” these by having scripts e.g. ./bin/cider ./bin/repl ./bin/kaocha apply them. However this is opaque to the tooling.
So cursive users have to manually select the specific aliases they want to translate the scripts to the aliases they want to use when they start a repl; and Emacs users need to run the right script with their own :cider alias appended and cider-connect to the repl.
Then a different project will have different combinations.
I find our alias definitions tend to become smaller over time, so they can be reused in other contexts — but then you end up with lots of small things that you have to remember to add back in when you start your REPL/tests/etc.
It’d be nice if tools deps aliases could also be sets of aliases… e.g. I’d really like to define the following:
:composite-aliases {:dev #{:prj/all :test :cljs :user/dev}
:prj/all #{:prj/customer-1 :prj/customer-2 :prj/customer-3 ,,,}
:prod/customer-1 #{:prod :prj/customer-1}
:prod/customer-2 #{:prod :prj/customer-2}}
#2019-11-1319:25seancorfieldAlias-aliases like that would be very handy sometimes.#2019-11-1319:34hiredmanAt work @seancorfield wrote a shell script that manages a lot of that stuff for us. Outside of work I rely pretty heavily on shell history for that kind of thing (type it in once, that search for it in history to run it again forever after), which has the added benefit of working for things outside of clojure too, but is not really sharable for a team.#2019-11-1319:46seancorfieldThat script knows about the high-level aliases and expands them into the set of low-level ones. It also knows how to run multiple clojure commands in sequence, to simulate lein/boot's ability to run multiple commands in one go#2019-11-1320:03hiredmanI think you might be better off creating coarser aliases. It seems to often be the case then when you add a meta language to a system (in this case aliases in deps are an aggregate built on top of clojure functions, but you can draw a similar analogy with things like di systems) the meta language starts out providing a coarse aggregate of base level functionality, and then overtime becomes finer overtime until it corresponds so closely with the base level that you lose the aggregate functionality and start building a new meta level (:composite-aliases) to try and regain the aggregation#2019-11-1409:18rickmoynihan> I think you might be better off creating coarser aliases
Yeah, I’ve tried that too. That’s essentially what the :prj/all alias is. It’s an alias that adds all (customer) project dependencies to the classpath; and it is essentially the union of all the customer aliases.
I find it becomes hard to then know what deps/paths/config belong to which alias though… e.g. I’m a believer that in the repl you should have both :dev-tooling and :test stuff available. But :test may also want to be included in the alias the CI uses to run the tests.
The problem then is that the groupings become lost, and you need to resort to comments to group things which is far from ideal.#2019-11-1320:05hiredmanlike, maybe get rid of the separate alias for adding cider, add it to the class path for all the dev aliases, but add some kind of runtime configuration option so it only launches nrepl on developers machines that want it#2019-11-1320:08dominicm@rickmoynihan 1. copy and paste 2. Not including things like cider in projects, that's userland stuff#2019-11-1409:21rickmoynihanI agree cider is userland stuff. We treat it as such.
This adds a further complication that each user might define cider in a different alias, e.g. I have :cider-22 :cider-21.#2019-11-1320:10dominicmYou can probably configure cider to call clojure always as -A:cider <prompted arguments>.#2019-11-1409:28rickmoynihanYeah I think you can do that but often we need to run the JVM in a particular environment, with env vars etc set…
So I tend to prefer to cider-connect to an instance started in a terminal.
I’ve solved some of the issues here by starting the dev repl and dependent services, along with shadow-cljs --watch etc with overmind; which uses a Procfile to start stuff; and allows you to share env vars across services.
I then use direnv to share project wide, not just repl wide, configuration… e.g. a port/location of a shared database across services etc.#2019-11-1320:11dominicmDev/test/cljs would probably be copy and pasted into each other as much as they were mutually dependent.#2019-11-1320:11dominicmI'm not sure what's different between customers for you, so it's hard to say what I'd do#2019-11-1409:30rickmoynihanMainly just assets on the resource path; and occasionally customer specific reference data.#2019-11-1409:34rickmoynihanThough some of those assets are bundled into webjars… e.g. css skins.#2019-11-1409:46plexusI quite like having wrapper scripts for this, this is also why I promote adding a bin/kaocha to projects. That also gives you an opportunity to further reduce friction by doing extra setup steps, doing precondition checks (e.g. minimum java version), setting environment variables, passing along JVM options, ...#2019-11-1410:00rickmoynihan@plexus: Yeah we use kaocha; it’s great, thanks for that btw 🙂
Though I think adding bin scripts is a bit of a hack/workaround, as it’s opaque to introspection and tool support. Which is another thing projects (particularly internal ones) then need to document; often for each editor/environment in use on the team.#2019-11-1410:57plexusI agree, but I think that's exactly why they are great, they are executable documentation. I'd rather see bin/cider_repl / bin/proto_repl / ... then having one liners in the README (not too bad) or having them shared ad-hoc through back channels (more common). It removes friction and cognitive overhead for the most common cases. You can still pass on extra args to clojure, and if that doesn't cut it then run it with bash -x, copy the final command into your shell and take it from there.#2019-11-1410:59plexusit also means most developers are doing the same thing. I've been on too many projects where it turns out every single person runs their dev process slightly different from everyone else, causing all kinds of reproducibility issues.#2019-11-1411:33rickmoynihanTIL. nice tip:
> run it with bash -x, copy the final command into your shell and take it from there.#2019-11-1411:36rickmoynihan> I’ve been on too many projects where it turns out every single person runs their dev process slightly different from everyone else, causing all kinds of reproducibility issues.
I think this is a side effect tools.deps has encouraged, by doing less.
Expectations and workflows on lein projects were typically more uniform.#2019-11-1417:51seancorfieldIt's why we have a standardized build script that runs clojure, and also why we have a standardized "baseline" deps.edn which we control via CLJ_CONFIG in the build script.#2019-11-1417:52seancorfieldNo user-level aliases interfering. No possibility of different versions across the team.#2019-11-1417:52seancorfieldThe baseline deps.edn contains all the aliases that anyone needs.#2019-11-1510:54rickmoynihanSo assuming I was on your team for a second…
Presumably I’d need any personal aliases to be committed into that file? What about if I wanted to use classpath overrides etc, to stub a dep with a localised version etc? I like to use scope-capture, assuming noone did already would that need to be a conversation with the team, or would I just add it in as a dep into everyones :dev alias?
I’m curious how the build script works… Does it start a repl as well as build stuff? Does it accept aliases as arguments? Or does it manages alias groups? i.e. ./build repl cider would start it with cider ./build repl cursive suitable for cursive etc?
What environments do people on your team use? Does everyone use chlorine and nrebl like you? Or do folk also user cider, intellij etc?#2019-11-1510:57rickmoynihanYou have me thinking though, that there’s a benefit to the consistency wrt to transitive deps of always at least having things like the cider deps on the classpath even if not everyone uses it; it wouldn’t need to be loaded after all 🤔#2019-11-1511:13danierouxWe optimise for pairing. And therefore value consistency very high.
We have Makefiles. And a make repl creates an nRepl on a known port, and both Cider and Cursive is configured to connect to it. We also share configurations.#2019-11-1511:41dominicmCider doesn't really have deps#2019-11-1513:18rickmoynihanyeah they’re shaded by mr anderson — but not all tools/libs will do this.#2019-11-1514:27dominicmThen we must be careful in our selection#2019-11-1514:28dominicmPick development tooling that itself only has safe deps#2019-11-1514:29rickmoynihanif its always there yeah… if it’s something you can opt in to it doesn’t matter as much#2019-11-1514:32dominicmI would say it's the other way round 😊 if it is bundled with the project and is always present then it doesn't matter. If I load it as a user, then it ought to be stable so I know I'm not causing the reproducibility problems that were mentioned.#2019-11-1514:37rickmoynihanYou might be right but I was thinking more for stuff like:
https://github.com/clojure-goes-fast/clj-async-profiler
I tend to only include it one off when I need it; rather than always have it there… not sure what its deps are tbh.#2019-11-1516:45dominicmThat's a good example of something you only use sometimes#2019-11-1511:42dominicmTooling dependencies have to be carefully selected because of what you describe#2019-11-1511:43dominicmAs someone who likes to tinker and improve on my development process, I find the idea of not supporting personal aliases distasteful.#2019-11-1516:54seancorfieldNo one on our team uses CIDER or nREPL these days but there are still two aliases in our shared deps file for starting such things (old and new versions). #2019-11-1516:58seancorfieldYes, the build script takes a list of aliases to use. Yes, it can start a REPL, REBL, Socket REPL, and several other variants (including old and new nREPL servers).#2019-11-1815:26practicalliWhat does the S in -S option for the Clojure CLI tools stand for? The other option names seem to be fairly obvious in meaning when reading clojure --help. For example -A is aliases, -C is classpath, -R resolve, etc.
Or is -S just a generic 'catch all' prefix for all other commands?#2019-11-1815:46dharriganIf I was to hazzard a guess, I think it means script - as examining the clojure script file that controls the process, S is used for script commands (internal to the script) and also a tree function, in the script package in tools deps alpha.#2019-11-1815:46dharriganI could be 100% waaaay of the mark here ... 🙂#2019-11-1816:09alexmillerIt stands for "settings"#2019-11-1816:09dharriganaha!#2019-11-1816:09dharrigan🙂#2019-11-1816:09alexmillerso, yes a generic prefix to catch many settings#2019-11-1816:11alexmillerin particular, the command line has a mix of clj args, clojure.main args, and when used as a runner, your own clojure main args#2019-11-1816:31sogaiuat least now i have two mnemonics :)#2019-11-1818:26practicallimy brain can rest at ease now, thank you Alex.#2019-11-1823:32danielsz@alexmiller Hi Alex, the console-listener Var in clojure.tools.deps.alpha.util.maven appears to be an implementation of the TransferListener interface. However it isn't a complete implementation, and I am seeing errors to that effect.#2019-11-1823:32danielszReceiver class clojure.tools.deps.alpha.util.maven$reify__7307 does not define or inherit an implementation of the resolved method 'abstract void transferSucceeded(org.eclipse.aether.transfer.TransferEvent)' of interface org.eclipse.aether.transfer.TransferListener.#2019-11-1823:34danielszPlease note that the docs suggest to extend the abstract class AbstractTransferListener rather than implementing the interface. The abstract class ships with no-op callbacks so a partial extension is a-OK and no errors would occur.#2019-11-1823:41alexmillerThx#2019-11-2020:00dadairWhat's the recommended way for evaluating the dependency tree for deps.edn based projects (e.g., lein deps :tree)? I've tried clj -A:dev:test -Spom && mvn dependency:tree -Dverbose=true, but I just get an exception that maven can't resolve the project deps#2019-11-2020:01dadairI'm trying to find out if a dep somewhere is loading an old tools.reader that ends up spitting out some warnings in my project#2019-11-2020:02dadairI've used clj -A:dev:test -Stree, but wasn't sure if there is a more verbose one, as the -Stree approach only shows one dependency on tools.reader and it's definitely of a version that includes a fix to not have the warnings..#2019-11-2020:05nwjsmith@dadair do you happen to have cloverage in your test dependencies?#2019-11-2020:05alexmillercan you share the exception?#2019-11-2020:07dadair@nwjsmith yes, through kaocha-cloverage#2019-11-2020:07nwjsmithit contains an AOT-ed version of tools.reader, which doesn’t show up in the dependency tree#2019-11-2020:08alexmillerevil#2019-11-2020:08nwjsmithI have a commit in master which brings the dependency up-to-date (https://github.com/cloverage/cloverage/commit/882c21041384c76d397d6429780e10531de9dc40)#2019-11-2020:37dadairCannot target the sha via deps as cloverage doesn't provide a deps.edn#2019-11-2020:09nwjsmithbut a release needs to be cut for it (only @lvh can do that)#2019-11-2020:09nwjsmithand we should probably stop AOT-ing that dependency#2019-11-2020:09nwjsmithI’ve had problems with it too#2019-11-2020:09dadair@alexmiller Could be of note, I am running through WSL2 (Ubuntu 18.04)#2019-11-2020:10alexmillerlooks like missing clojars repo or something#2019-11-2020:14lvhHi: I think I mentioned this in a ticket somewhere but CI should be building snapshots#2019-11-2020:14lvhI’m on my way to the Conj right now so I can’t check#2019-11-2020:17nwjsmithOh! good call. @dadair you might get rid of those warnings by adding cloverage {:mvn/version "1.1.3-SNAPSHOT"} to your deps.edn.#2019-11-2020:20dadairDoesn't seem to be making a difference#2019-11-2020:21dadairConfirmed that the override is in place, just not working
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "3e5a5f48575a7e7a7b6d756a716e137a730d7f766d7f"}, :content ("[email protected]")}#2019-11-2020:24alexmillerIf cloverage has the other one and ends up first in the classpath it will win#2019-11-2020:25alexmillerWhether it’s first is pretty arbitrary#2019-11-2020:26alexmiller-Spath should be exactly what you’re running#2019-11-2020:29dadairhmm, using -Spath, cloverage is on the snapshot version (1.1.3-SNAPSHOT), and tools.reader is at the fixed version 1.3.2, yet the warnings are still printing..#2019-11-2020:37miikkaFWIW, cloverage-1.1.3-20191022.214104-8.jar still contains AOT'd tools.reader#2019-11-2020:39miikkaOh, right, as it should#2019-11-2022:12alexmilleryou can take clj out of the loop by doing java -cp <the-path-from-Spath> clojure.main <your-main-args> and then feel free to reorder / edit the cp to try things#2019-11-2106:33miikkaThe Cloverage snapshot still seem to cause the problem. I get the warning with the following classpath by running java -cp ... clojure.main -e "(require 'clojure.tools.reader)":
/Users/miikka/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/miikka/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/miikka/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/miikka/.m2/repository/cloverage/cloverage/1.1.3-SNAPSHOT/cloverage-1.1.3-SNAPSHOT.jar
#2019-11-2106:52miikkaI'm now wondering if :refer-clojure :exclude and AOT actually work correctly together.#2019-11-2106:53miikkaShould be easy to test, but I have to do something else now.#2019-11-2113:16dominicmDepending on coverage goals, one could avoid the monroe shebang: https://www.crystae.net/posts/2019/11/08/two-shebang-papercuts/#2019-11-2114:22alexmillerclj 1.10.1.489, tools.deps.alpha 0.8.599
* TDEPS-143 - Fix: use aliases when generating pom with -Spom
* TDEPS-127 - Add: include pom resource directories in dep paths in -Spom (thanks Miikka Koskinen)
* TDEPS-140 - Add: support for Maven server HTTP header properties (thanks Arne Brasseur)
* Fix: Use of multiple aliases in Windows clj script
* Add -Strace for debugging dep expansion#2019-11-2119:06mssgetting a weird error when attempting to use depstar to build an uberjar:
Execution error (FileAlreadyExistsException) at com.sun.nio.zipfs.ZipFileSystem/createDirectory (ZipFileSystem.java:422).
META-INF/versions/9/
#2019-11-2119:07mssI can’t tell if I’ve configured something incorrectly in intellij, if I’m missing an option in building the classpath, or something else entirely. not much is coming up when googling around about a META-INF related FileAlreadyExistsException#2019-11-2119:09mssif people have a recommendation for tooling to use to build standalone jars, I’m all ears as well#2019-11-2119:10mssinterestingly, using depstar to build a slim jar and attempting to run that jar results in a Error: Invalid or corrupt jarfile error#2019-11-2119:39seancorfield@mss I'm at Conj so I won't be able to look at that until next week but if you can put a repro case up on github and create an issue against depstar with a link to your repro case, that would really help #2019-11-2119:41seancorfieldYou can't run a "slim jar" - you can only use that as a library (since it won't contain any dependencies).#2019-11-2121:40jfntnIs there a supported approach for merging deps from a parent directory in a monorepo?#2019-11-2123:09seancorfield@jfntn At work we use the CLJ_CONFIG environment variable to point to the "master" deps file so that it gets combined with the subproject's deps file within our monorepo.#2019-11-2123:10seancorfieldcd subproject
CLJ_CONFIG=.. clojure
(our master deps file is in ../versions not .. but the same principle applies)#2019-11-2218:09jfntnInteresting but that would prevent using user defined aliases etc right?#2019-11-2218:10jfntnAlso how do you enforce the env var, do you have to use a makefile or some additional env tooling?#2019-11-2223:02seancorfield@jfntn Yes, a build shell script that runs clojure one or more times so we can have a single command run multiple tasks, all with CLJ_CONFIG set. And, yes, no user-defined aliases can interfere with our build process. Any developer tooling is available via aliases in that master deps.edn file.#2019-11-2312:20daveliepmannI borked a copy/paste in my deps.edn and briefly had a dependency pointing to a different git repo. In other words the key is library X but the git repo coordinate is library Y. What's the expected behavior in this situation?
org.clojure/tools.deps.alpha {:git/url ""
:sha "e5f3d9e6839191f1e37ddfa51cf442b2d5403ff3"}#2019-11-2312:33alexmillerThere’s no way for tda to tell #2019-11-2314:14dominicmhttps://github.com/clojure/tools.deps.graph 👀#2019-11-2314:18kszaboI am getting:
Error building classpath. Could not find artifact org.clojure:tools.deps.graph:jar:0.1.5 in central ()
#2019-11-2314:23dominicmThe repo was empty a moment ago#2019-11-2314:25seancorfieldMaven Central can take several hours to list a new release. I suspect that this is something Alex will debut in his talk in a few hours. #2019-11-2314:27kszaboI wish I could be there 😢#2019-11-2314:29seancorfieldAll the talks are showing up on YouTube already!#2019-11-2314:36alexmillerthat was a dummy version - it's up there now at 0.1.14#2019-11-2314:49dominicmThe conj is still on?! Yay, more to watch#2019-11-2314:49dominicmHaving the videos go up live is fantastic for those of us across the pond#2019-11-2315:48sogaiueven on the same lilypad, it's much appreciated 🙂#2019-11-2315:54seancorfieldEven if you are actually here it's great. I missed a talk yesterday but was able to watch it online later before going out for dinner! #2019-11-2316:07sogaiunice!#2019-11-2314:32alexmillershhh#2019-11-2415:29andy.fingerhutAlex, you mentioned what I think you called logically inconsistent deps that one occasionally comes across in the deps of projects. Is there an existing option to clj/clojure, tools.deps, or some other tool that tells one if a project has such logically inconsistent deps?#2019-11-2415:36alexmillerNo, it will just muddle through#2019-11-2415:49andy.fingerhutSure. Just wondering for yours (and others) debug purposes if something like that existed, as a machine-detectable property that can signify that here might be dragons.#2019-11-2419:49lreadEnjoyed your talk @alexmiller https://youtu.be/7CM7Ef-dPWQ thanks!#2019-11-2420:26lvhI want to run a tool specified in a deps.edn file as an alias (i.e. with a :main-opts), but with a multiple sets of command line arguments to debug an issue down to a particular combination of arguments. do I just use clojure.main repeatedly or is there a smarter way to do that?#2019-11-2613:41unbalancedGood morning! Darn conj has me so full of ideas now 😮
So what does it take to hook into the tools-deps machinery? Is there a general primer-faq? Reason I'm asking is b/c I'm working with Chris Nuernberger on libpython-clj and I think it would be a very good idea to be able to specify Python deps declaratively. Is tools-deps designed to be extensible like that or should we use a seperate .edn file inspired by deps.edn?#2019-11-2613:42dominicmIt is extensible, but the clj tool currently doesn't define a way for extensions to be loaded#2019-11-2613:43vlaaadI thinks it's not extensive in that direction. it's a tool to resolve java dependency and build java classpath, it probably can't be extended for other ecosystems such as python or js. they are too different.#2019-11-2613:49alexmillerclj is about making classpaths so the question is whether there is any use in putting python stuff on a classpath#2019-11-2613:51unbalancedSo the use case here is for library authors using the libpython-clj tool#2019-11-2613:51alexmillerYou could use just the resolve-deps part with custom providers to do the transitive expansion#2019-11-2613:52unbalancedThere's a chicken/egg problem -- basically the Python VM needs to be loaded before any libpython-clj libraries can be pulled in#2019-11-2613:52alexmillerUnclear if that’s worth doing via tolls deps#2019-11-2613:53unbalancedOf course! I was't sure if it was designed for that or not#2019-11-2613:57alexmillerI don’t know enough about python stuff to tell but the provider extensions are pretty generic so it could probably be made to work, just not sure if it’s worth doing#2019-11-2614:40orestisPython has literally a PYTHONPATH which is pretty similar to the classpath. But in practice I think people don’t do that. The Python community changed the “de-facto” dependency manager 4 times in the last 5 years already (I haven’t looked deeply into python rencetly, might be wrong).#2019-11-2615:35dominicmI think they still use the same repo though don't they?#2019-11-2615:37dominicmThe real question is:
- How does libpython-clj search for python code? It could search the classpath, jruby can.
- How does graal python find python code?
Those are the two consumers I can think of.#2019-11-2817:55souenzzoHow do I import https://mvnrepository.com/artifact/org.apache.cxf/cxf/3.3.4 in deps.edn?
I'm trying org.apache.cxf/cxf {:mvn/version "3.3.4" :extension "pom"}
It add a .pom to my classpath, but I can't import anything
(filter
#(string/includes? % "cxf")
(string/split(System/getProperty "java.class.path") #":"))
=> ("/home/souenzzo/.m2/repository/org/apache/cxf/cxf/3.3.4/cxf-3.3.4.pom")
#2019-11-2817:58dominicmhttps://maven.apache.org/pom.html#Dependencies type is documented here for others who have never heard of this 🙂#2019-11-2818:00dominicmokay, https://maven.apache.org/ref/3.6.3/maven-core/artifact-handlers.html is way more interesting. tools.deps maybe doesn't support pom extensions because they're not added to the classpath.#2019-11-2818:06souenzzoOkay. So I will not be able to use this dep at this time?#2019-11-2818:06seancorfieldYou'll need to manually specify all the deps that the POM specifies I think.#2019-11-2818:07seancorfieldI thought there was a way for t.d.a. to pull in things via a pom dep but I can't think what it is right now...#2019-11-2818:14dominicmYou definitely don't want it on the classpath, that's for sure 🙂#2019-11-2818:16souenzzoNice to know#2019-11-2818:15souenzzoSolved by use https://mvnrepository.com/artifact/org.apache.cxf/cxf-core/3.3.4#2019-11-2822:36sogaiui am seeing what looks like extra strings "/main" and "/java" being tacked on to a couple of the non-jar items of a classpath stored in some files in .cpcache.
specifically, for: https://github.com/FundingCircle/fc4-framework/tree/master/tool#running-the-tests , running clj leads to .cp and .libs files containing:
• .gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/a1b1b0903ae5ba56eaada8496ed3785a53d8cc06/src/main
• .gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/a1b1b0903ae5ba56eaada8496ed3785a53d8cc06/src/java
looking at the repository for the library in question (https://github.com/tatut/clj-chrome-devtools), i see a src directory but no subdirectories named "main" nor "java".
there is a deps.edn file, but it has no :paths key
is this expected behavior?#2019-11-2822:43seancorfieldThe :paths in fc4-framework/tool/deps.edn has src/main so that will propagate I believe. Not sure where src/java is coming from. What's in your ~/.clojure/deps.edn file?#2019-11-2822:46seancorfieldHmm, perhaps not. I just cloned the repo and ran clj -A:dev:test in the tool folder and when I look in .cp I do not see src/main or src/java#2019-11-2822:46seancorfield(! 858)-> cat .cpcache/1648314215.cp | tr ':' '\n'|fgrep gitlib
/Users/sean/.gitlibs/libs/expound/expound/297e0812eb77ae3aa8f30e823f8bdbbb4650fa28/src
/Users/sean/.gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/a1b1b0903ae5ba56eaada8496ed3785a53d8cc06/src
#2019-11-2822:47seancorfield^ @sogaiu Maybe this is from your user-level deps.edn file?#2019-11-2822:53sogaiu@seancorfield thanks for checking
my ~/.clojure/deps.edn file is basically a slightly modified version of your public one 🙂 grepping for java in it doesn't turn up any culprits (just the javafx deps for rebl)
i see my clojure-tools is 1.10.1.447 though, may be i should update that and see if it makes any difference#2019-11-2822:57seancorfield(! 861)-> clj -Sdescribe
{:version "1.10.1.492"
:config-files ["/usr/local/Cellar/clojure/1.10.1.492/deps.edn" "/Users/sean/.clojure/deps.edn" "deps.edn" ]
#2019-11-2822:59seancorfield(after updating your CLI, remember to delete .cpcache to make sure it regenerates it)#2019-11-2823:02sogaiuthanks -- what you said about propagating seems to be happening. i failed to mention i was not on the master branch. the branch i'm on has "src/java" as one of its paths.#2019-11-2823:03seancorfieldAh, interesting. With the latest CLI I see src/main in .cp but only for the tool repo itself, not for any transitive deps.#2019-11-2823:03sogaiufwiw, the branch i'm trying is native-image#2019-11-2823:03seancorfieldSo if you update and re-generate and no longer see src/main/`src/java` then it sounds like a bug that got fixed since .447#2019-11-2823:03sogaiuright -- will work on that#2019-11-2823:05seancorfieldI switched to that branch, deleted .cpcache, and re-ran clj -A:dev:test and now I see src/java in .cp but only for the and it does not propagate into the git deps.#2019-11-2823:10sogaiuthanks for checking -- hopefully upgrading will fix it 🙂#2019-11-2823:16sogaiu@seancorfield upgrading seems to have fixed things -- thanks a lot for your help 🙂#2019-11-3020:40Nolantools.deps.graph is wonderful. the missing piece of rooting out any unexpectedness in deps projects. 🙏 x`Inf` @alexmiller#2019-12-0204:26mike_ananevHi! Is there any way to build WAR using tools deps? I know only leiningen plugin for building war, but it is not convenient to use both tools.#2019-12-0204:58seancorfield@mike1452 I don't know if any of these tools can build WAR files https://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging#2019-12-0204:58seancorfieldBut a WAR is "only" a JAR/ZIP with some additional metadata files in it...#2019-12-0205:06sogaiuthis was once claims to: https://github.com/EwenG/badigeon#2019-12-0205:07sogaiuperhaps this part of the sample is relevant? https://github.com/EwenG/badigeon/blob/master/sample/badigeon/sample.clj#L200#2019-12-0208:15dominicmSomeone requested it for pack, but I never got round to it because I'm not very familiar with war. I think it should be straightforward using the internal API, because the layout is very similar to the other supported types.#2019-12-0221:26didibusHow does tools.deps resolve conflicts for git deps and local deps ?#2019-12-0221:36seancorfieldI believe it can't resolve conflicts across different types of deps? For two git deps it can resolve to the "most recent SHA" I think. Not sure how it would handle two conflicting local deps.#2019-12-0223:09alexmillerGit - newest wins, local - nothing implemented now but I think that is rare#2019-12-0221:41seancorfieldI can't find a JIRA issue for this. @alexmiller is there an issue for this that I'm just not seeing? Or is this one of those "known issues" that hasn't gotten as far as JIRA yet?#2019-12-0223:10alexmillerThere is a ticket similar to this but I could not repro it last week, hoping to clarify soon#2019-12-0221:43seancorfield@didibus It feels like a local dep should always be preferred over a git dep which in turn should be preferred over a Maven/Clojars dep but maybe that's just my misguided intuition...#2019-12-0222:17didibusYa, it can be tricky. I wonder if introducing a deps concept of version could help, like if you declared the version in deps.edn#2019-12-0222:17didibusNow it could be used to compare across revolvers#2019-12-0222:18didibusAnother thing could be to allow to specify a version when the resolver has no concept of it. So I declare a local deps, maybe I could also tell tools.deps what version it is#2019-12-0222:19didibusBut that one would seem more confusing#2019-12-0223:06alexmillerYeah, not planning to do that. Most Maven artifacts include their shas (central enforces, not sure if Clojars does) which gives you comparison basis w git deps (even across forks)#2019-12-0223:08alexmillerNot all cases will be resolvable - in that case it’s an error and you (human) must resolve by picking one via top dep or override#2019-12-0223:27seancorfield@alexmiller Does that mean that if you have a :git/url and a :mvn/version and the pom.xml for the latter includes a <tag> that t.d.a. currently resolves that to a SHA and figures out whether it's before/after the :git/url's :sha? Or is that just speculative "t.d.a. could maybe do this perhaps"?#2019-12-0223:28seancorfield(I'm guessing that it's speculative and in many cases it could not resolve that anyway because there isn't always valid tag/scm information in the POM anyway)#2019-12-0223:47alexmillerThere usually is scm info#2019-12-0223:47alexmillerI have not yet implemented this but all the pieces are already there#2019-12-0300:01didibusI see, so local deps always wins, and in the case of conflicting local deps it would error.#2019-12-0300:01didibusBut maven and git can both use the commit sha to find out which one is newest#2019-12-0300:09didibusI'm not seeing anything scm related or git related in the manifest spec: https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Manifest_Specification there's a version-number field, but that assumes digits#2019-12-0300:10seancorfield@didibus it would be in pom.xml not the manifest#2019-12-0300:11seancorfieldfor example
<scm>
<url></url>
<connection>scm:git:</connection>
<developerConnection>scm:git:@github.com/seancorfield/next-jdbc.git</developerConnection>
<tag>05a6d8a47fcea7dc2928e5e2c176310fb7205e0c</tag>
</scm>
(from next.jdbc's pom.xml that is used for deployment to Clojars)#2019-12-0300:12didibusI thought it was common to put tag to HEAD ?#2019-12-0300:13didibusAlso, what does maven do with that? Is the pom.xml uploded to Clojars as well?#2019-12-0300:15seancorfieldYes, the pom.xml is what tells Clojars how to link to the commit that is tied to the release.#2019-12-0300:16seancorfieldIt can either be a specific tag or a SHA.#2019-12-0300:16didibusI see#2019-12-0300:16seancorfield(I happen to use SHAs, just because I have a deploy script that does that automatically)#2019-12-0300:17didibusSounds like days worth of fun debugging conflicts to be honest 😛 But hopefully it mostly works itself out in practice#2019-12-0300:17seancorfieldWhy would you think that?#2019-12-0300:17seancorfieldClojars shows which commit is referenced in pom.xml#2019-12-0300:19didibusWell, maybe I'm wrong. But I can see the tag not mapping to the correct version, and issues incurring. Though I guess there's nothing to do about that#2019-12-0300:19seancorfieldSure, you can "lie" in pom.xml if you don't have some automated release process...#2019-12-0300:20didibusAnd what does happen if you set HEAD as the tag?#2019-12-0300:20didibusIs that automatically replaced by the HEAD sha at time of deploy?#2019-12-0300:20seancorfieldDeployment is independent of Git (so, no).#2019-12-0300:21seancorfield(at least, that is my understanding)#2019-12-0300:22seancorfieldCertainly, you can mess up the <scm> part of pom.xml -- when I first started publishing to Clojars via clojure-generated POM files, I messed up several times. Hence, automation 🙂#2019-12-0300:22seancorfieldcljdoc also relies on pom.xml being accurate so it can pull the correct version of your repo from GitHub to generate docs that match the release artifact...#2019-12-0300:23didibusI mean, the example on Maven shows HEAD as tag: https://maven.apache.org/pom.html#SCM#2019-12-0300:23didibusIt says: tag: Specifies the tag that this project lives under. HEAD (meaning, the SCM root) should be the default.#2019-12-0300:27didibusAnyways, it does seem more standard then I thought. It should be a pretty nice way to handle cross resolver conflicts.#2019-12-0300:28didibusI guess a local deps could also use the pom.xml found in the directory for conflict resolution ?#2019-12-0300:31alexmillerit could if there is one#2019-12-0300:31alexmilleralthough I think the chances that value is meaningful+correct in a local dep is low#2019-12-0300:32alexmillervs the rev in an artifact, which is almost always updated during deployment#2019-12-0300:32alexmilleropen question what to do for a local dep that also happens to be a git repo#2019-12-0300:33alexmillerI would need to think about that for a while and make some tables#2019-12-0300:35didibushammock#2019-12-0300:35alexmillerthere are requests for git deps on the file system too (which we don't support right now) so that intersects#2019-12-0300:37didibusThe world of dependency management is hard. I've lost so many hours of my life managing dependencies 😛 At least Clojure libs rarely break anything, so which version ends up chosen doesn't matter as much 😛#2019-12-0300:39alexmilleryeah, I've been doing this stuff for 22 years in the Java universe. those early years pre-Maven were kinda weird.#2019-12-0300:40didibusI do love how tools.deps resolves conflicts by picking latest though. That's a huge improvement for me with regards to Maven.#2019-12-0300:40seancorfieldSomehow I managed to completely skip Maven until I got to Clojure... 👀#2019-12-0300:42alexmillermanaging your deps manually is a very instructive process (not very efficient in the long run, but doing it a little really gives you an appreciation)#2019-12-0300:42alexmillerI used (and wrote) a lot of bespoke tools pre-Maven#2019-12-0300:44didibusI never lived in the pre-maven world, for the better I guess#2019-12-0310:16niclasnilssonadding libs on the fly in the repl? If I’m not mistaking the commit pointed to in http://insideclojure.org/2018/05/04/add-lib/ is gone, and github says there is no add-lib function or a repl ns in the repo. Any ideas on how people do this nowadays?#2019-12-0310:18dharrigan@niclasnilsson If you look at this: https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#2019-12-0310:18dharriganand in particular line 139#2019-12-0310:18dharriganyou'll see how it is used 🙂#2019-12-0310:36niclasnilsson@dharrigan, thanks (I wonder why github didn’t find add-lib in that very repo though?) . But, I don’t get it work though. Installing the deps worked, but I get a NullPointerException when trying to add a lib. Anything you recognize?
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'honeysql {:mvn/version "0.9.8"})
Execution error (NullPointerException) at clojure.tools.deps.alpha.util.maven/make-session (maven.clj:173).
null#2019-12-0310:39alexmilleradd-lib exists only on a branch, has never been integrated on the main line#2019-12-0310:40niclasnilsson@alexmiller, yep, understood. I used the sha in Sean’s deps.edn right now though#2019-12-0310:44alexmillerThat sha is on the add-lib branch which typically lags master#2019-12-0310:41niclasnilsson(when I get the NPE)#2019-12-0310:41alexmillerYou should never see that error but I have seen people report this. I’ve never been able to repro#2019-12-0310:42niclasnilssonInteresting. Any special investigation I can undertake on my env?#2019-12-0310:43alexmillerIt should be impossible by my understanding of the code so clearly some assumption I have is wrong#2019-12-0310:43niclasnilssonEven more interesting 🙂#2019-12-0310:44dharrigan@niclasnilsson seems to work for me!#2019-12-0310:44dharriganuser=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'honeysql {:mvn/version "0.9.8"})
true#2019-12-0310:44dharriganTry deleting your .cpcache?#2019-12-0310:45niclasnilssonGood idea. Will try. (But I’ll save it some place else)#2019-12-0310:45alexmillerShould t be using cpcache#2019-12-0310:45dharrigank#2019-12-0310:45alexmillerThat’s for caching classpaths but you’re not making those here, you’re dynamically updating runtime#2019-12-0310:46alexmillerWhat repl env are you in?#2019-12-0310:46dharrigan👍#2019-12-0310:46alexmillerNot all repls have the right classloader setup for this to work#2019-12-0310:49niclasnilssonClearing .cpcaches didn’t work#2019-12-0310:50alexmillerI’d be shocked if it did :)#2019-12-0310:50niclasnilssonrepl: Vanilla clj#2019-12-0310:51niclasnilsson(no socket, no nrepl)#2019-12-0310:51alexmillerWell, don’t know then#2019-12-0310:52niclasnilssonThe docs say “Note that for successful use, you must be in a REPL environment where a valid parent DynamicClassLoader can be found in which to add the new lib urls.“, but I suppose that’s the case when I just fire up clj?#2019-12-0310:52alexmillerYes#2019-12-0310:52niclasnilssonOk#2019-12-0310:54niclasnilssonHmm… It works if I have a deps.edn only contaning
{:deps
{org.clojure/tools.deps.alpha {:git/url ""
:sha "148cab8f154711f8495bd632ebdc53a14dff7f09"}}}
#2019-12-0310:55niclasnilssonBut not in my project, so likely something in there messes it up. I’ll reduce my deps.edn and see if I can find it.#2019-12-0310:57dharriganyou can do clojure -Stree to get a list#2019-12-0310:57alexmillerIf you have something that doesn’t work now, I’d take that#2019-12-0310:58dominicmI had a repro for this involving dependency conflicts#2019-12-0310:59dominicmhttps://github.com/juxt/pack.alpha/issues/64 oz transitively brings in a different maven package.#2019-12-0311:00dominicmThe session lookup fails because its the wrong class#2019-12-0311:00dominicmIt's one from a different dependency#2019-12-0311:01dominicmIt's not specific to the add lib branch, it's just the only time users interact with maven outside of the clj internal classpath#2019-12-0311:11niclasnilsson{:deps
{metasoarous/oz {:mvn/version "1.6.0-alpha5"}
org.clojure/tools.deps.alpha {:git/url ""
:sha "148cab8f154711f8495bd632ebdc53a14dff7f09"}}}
and
(require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
(add-lib 'honeysql {:mvn/version "0.9.8"})
causes the NullPointerException.
Unfortunately oz pulls in half the world…#2019-12-0311:13niclasnilssonAh, I see @dominicm already pointed out oz and the cause.#2019-12-0311:15dominicm@alexmiller let me know if there's anything I can do to help with explaining or documenting this. I spent a lot of effort debugging it, and I hate for that to be a waste.#2019-12-0311:15alexmillerOh interesting, I had not seen all that debugging before#2019-12-0311:15alexmillerDo you have jira access?#2019-12-0311:15dominicmYou helped me with it 😜#2019-12-0311:15dominicmI do, yes.#2019-12-0311:16alexmillerIf you could file a tdeps ticket, that would be great for tracking#2019-12-0311:16alexmillerI don’t think there is one#2019-12-0311:20alexmillerCertainly oz should not be publishing an uberjar, that’s evil#2019-12-0311:21alexmillerI’d like to be able to fail better in tdeps though#2019-12-0311:26dominicmoz shouldn't be doing an uberjar, but this is unrelated to that.#2019-12-0311:27dominicmthe problem is that maven-aether-resolver and maven-resolver-provider conflict. I think oz gets the former through pomegranate, but icr right now, will double check before I file the jira.#2019-12-0311:27dominicm@alexmiller I think I did mention the resolution in this channel, is there a better place to mention that kind of thing to make sure you don't miss it next time?#2019-12-0311:34alexmillerJira#2019-12-0311:29dominicmAs a total aside, on my wishlist if for maven to support "provides", as in "juxt/tools.namespace" provides "org.clojure/tools.namespace", so there should only be one of them in the classpath (same for things where a -complete artifact is provided such as antlr. There's N artifacts and a "bundled" artifact too.)#2019-12-0321:55markbastianI'm having a deps issue with (drumroll, please), jackson. I have com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"} in my :deps map, but when I do an ion-dev push it reports a conflict of com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.8"}. This manifests itself when my ns gets loaded with the error "com.fasterxml.jackson.core.exc.InputCoercionException," which is a 2.10 addition. I tried adding the library to :override-deps instead, without success. Any tips on what I am doing wrong (besides needing jackson)?#2019-12-0322:00alexmillercan you share your ion-dev alias?#2019-12-0322:00alexmillerspecifically, is it using :extra-deps or :deps?#2019-12-0322:00alexmillerand if it's using :extra-deps, can you switch it to :deps? (need newish version of clj for that to work)#2019-12-0322:01markbastianSure, latest attempt:
{:aliases
{:ion-dev
{:deps {com.datomic/ion-dev {:mvn/version "0.9.247"}}
:main-opts ["-m" "datomic.ion.dev"]}}
:paths ["src" "resources"]
:deps {com.cognitect/anomalies {:mvn/version "0.1.12"}
com.datomic/client-cloud {:mvn/version "0.8.78"}
com.datomic/ion {:mvn/version "0.9.35"}
org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/data.json {:mvn/version "0.2.6"}
metosin/ring-http-response {:mvn/version "0.9.1"}
ring/ring-defaults {:mvn/version "0.3.2"}
ring/ring-json {:mvn/version "0.5.0"}
metosin/jsonista {:mvn/version "0.2.5"}
metosin/reitit {:mvn/version "0.3.10"}
javax.xml.bind/jaxb-api {:mvn/version "2.4.0-b180830.0359"}
}
:override-deps {com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"}
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.10.1"}}
:mvn/repos {"datomic-cloud" {:url ""}}}#2019-12-0322:02alexmillerso the error is not during push, it's when your code is run in the ion#2019-12-0322:03alexmillerright?#2019-12-0322:03alexmillerthat :override-deps in your deps.edn isn't doing anything - that only works in an alias#2019-12-0322:04alexmillerbut you can just add those to your main :deps#2019-12-0322:04markbastianRight, I get a warning when I push stating that I have the dependency conflict on 2.9.8 and the error is at runtime. I am assuming this is because the ion is using the older version without the InputCoercionException class.#2019-12-0322:04markbastianYeah, that override-deps was just my attempt to get the override to work. It was in deps before.#2019-12-0322:04alexmiller{:aliases
{:ion-dev
{:deps {com.datomic/ion-dev {:mvn/version "0.9.247"}}
:main-opts ["-m" "datomic.ion.dev"]}}
:paths ["src" "resources"]
:deps {com.cognitect/anomalies {:mvn/version "0.1.12"}
com.datomic/client-cloud {:mvn/version "0.8.78"}
com.datomic/ion {:mvn/version "0.9.35"}
org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/data.json {:mvn/version "0.2.6"}
metosin/ring-http-response {:mvn/version "0.9.1"}
ring/ring-defaults {:mvn/version "0.3.2"}
ring/ring-json {:mvn/version "0.5.0"}
metosin/jsonista {:mvn/version "0.2.5"}
metosin/reitit {:mvn/version "0.3.10"}
javax.xml.bind/jaxb-api {:mvn/version "2.4.0-b180830.0359"}
com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"}
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.10.1"}}
:mvn/repos {"datomic-cloud" {:url ""}}}
#2019-12-0322:06alexmillerok, well then I would send you to #datomic - this is not the first time they've seen something like this, but I don't know the recommended path forward. I think you're right on with what the problem is.#2019-12-0322:06markbastianI'll give that shot and report the output. I think I had done that before. BTW, the jaxb addition was because of this warning when I do the ion push: [s3-transfer-manager-worker-6] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant#2019-12-0322:06markbastianCool, let me give this a shot and I'll let you know what happened then bounce over to datomic.#2019-12-0322:06alexmilleryeah, that's likely due to being on java 9+#2019-12-0322:07alexmillerit's included in java 8#2019-12-0322:07alexmillerbut they've been shedding all those j2ee libs from the jdk#2019-12-0322:07markbastianIs the datomic runtime 8 or 9?#2019-12-0322:07alexmillerdunno#2019-12-0322:07alexmillerI mean, it must be 8 or 11#2019-12-0322:07alexmiller9 and 10 are dead and I can't imagine they're using those, so I'd guess 8#2019-12-0322:09markbastianFull output:
Downloading: com/datomic/java-io/0.1.11/java-io-0.1.11.pom from
(cognitect.s3-libs.s3/upload "datomic-code-d70d7189-fa46-4cd5-8004-423b714e486b" [{:local-zip ".datomic-ions/datomic/apps/ion-repl/stable/ff0ed784b940e81fa02f8a0808a3fc06719bfc1b.zip", :s3-zip "datomic/apps/ion-repl/stable/ff0ed784b940e81fa02f8a0808a3fc06719bfc1b.zip"}] {:op :push})
[s3-transfer-manager-worker-6] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant
{:rev "ff0ed784b940e81fa02f8a0808a3fc06719bfc1b",
:deploy-groups (ion-repl-Compute-1TPY2S14U7NT9),
:dependency-conflicts
{:deps
{com.cognitect/transit-java #:mvn{:version "0.8.311"},
org.clojure/clojure #:mvn{:version "1.10.0"},
commons-codec/commons-codec #:mvn{:version "1.12"},
com.datomic/client-api #:mvn{:version "0.8.37"},
com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.8"},
com.datomic/client #:mvn{:version "0.8.86"},
org.msgpack/msgpack #:mvn{:version "0.6.10"},
com.datomic/client-impl-shared #:mvn{:version "0.8.69"},
com.cognitect/transit-clj #:mvn{:version "0.8.285"},
com.cognitect/s3-creds #:mvn{:version "0.1.23"},
com.datomic/client-cloud #:mvn{:version "0.8.80"},
com.amazonaws/aws-java-sdk-kms #:mvn{:version "1.11.479"},
com.amazonaws/aws-java-sdk-s3 #:mvn{:version "1.11.479"}},
:doc
"The :push operation overrode these dependencies to match versions already running in Datomic Cloud. To test locally, add these explicit deps to your deps.edn."},
:deploy-command
"clojure -A:ion-dev '{:op :deploy, :group ion-repl-Compute-1TPY2S14U7NT9, :rev \"ff0ed784b940e81fa02f8a0808a3fc06719bfc1b\"}'",
:doc
"To deploy to ion-repl-Compute-1TPY2S14U7NT9, issue the :deploy-command"}#2019-12-0322:10markbastianI'll try the deploy, but I think it's going to be broken. I'll try #datomic. Thanks!#2019-12-0322:10alexmilleryeah, I mean it's telling you that in the result there#2019-12-0322:10markbastianRight. "We're using this library that hasn't got a class you're going to need."#2019-12-0322:52Alexis Vincent@alexmiller Hi there Alex, thanks for building/maintaining deps. And all the rest you do for the clj community. Much appreciation!
Had a couple of questions.
1. I’m trying to use tools.deps as a lib. But have an issue using resolve-deps . One of the deps.edn files uses a package from a mvn repo, where the repo is declared in the deps.edn file, and where the cli picks it ub correctly. But it blows up for me when using the lib with:
Could not find artifact io.confluent:kafka-schema-registry-client:jar:5.3.0
in central ()
#2019-12-0322:53Alexis Vincentim calling it using
(-> {:deps {'root/root {:local/root "<path to root project>"}}
:mvn/repos mvn/standard-repos}
(deps/resolve-deps nil
{:trace true})))#2019-12-0322:55Alexis Vincentthe required file includes the mvn repo as
:mvn/repos
{"confluent" {:url ""}}#2019-12-0322:58Alexis Vincent2. I have a case where that file adds a jar to its paths. But when using juxt’s pack, it thinks the jar lives at one of the transative local/root dependencies dirs. I dont know if this is a pack problem or something with tools.deps. But im guessing deps.#2019-12-0408:01dominicmI don't fully follow this, sorry. Can you expand?#2019-12-0323:00Alexis VincentHow do I best give you info to help fix the bug? I’ve managed to work around it for now using depstar#2019-12-0403:38alexmiller@mail024 so the mvn/repos declaration is in a transitive deps.edn? I suspect that's probably not handled right now#2019-12-0403:46alexmillerwe don't have a ticket for that but same issue exists with pom deps (https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-46)#2019-12-0403:56alexmillerI added a note there#2019-12-0414:54FiVoT#2019-12-0414:54FiVoThe link https://clojure.github.io/tools.analyzer.jvm/spec/quickref.html in the tools.analyzer.jvm README is broken.#2019-12-0414:55FiVoA sorry wrong channel.#2019-12-0415:15alexmillerworks for me?#2019-12-0415:16alexmillerI guess he fixed it#2019-12-0415:16alexmillerI must have accidentally nuked it yesterday#2019-12-0415:16FiVoYes it's up again.#2019-12-0620:04dominicmI just responded to this pack issue which I think is a tdeps bug, but I don't know much about maven/pom.xml, so it might be helpful to @drewverlee if someone who was more familiar took a look: https://github.com/juxt/pack.alpha/issues/68#issuecomment-562717952
Essentially it doesn't seem like the sourceDirectories in pom.xml are being read.#2019-12-0620:13drewverleeOn my phone atm, I'll be back home on Sunday and I'll look into this.#2019-12-0620:13drewverleeThanks for looking into the issue.#2019-12-0620:15dominicmLooks like I'm wrong, I'll take another look in a minute :)#2019-12-0620:09alexmillerI don't understand what the expected and actual behavior is?#2019-12-0620:09alexmillerI tried
clj -Spath -Sdeps '{:deps {integrant {:git/url "" :sha "c673b85130e553feec6d4c5d2d1ec773a49c929c"}}}'
#2019-12-0620:09alexmillerand got
src:/Users/alex/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/alex/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src:/Users/alex/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src/main/clojure:/Users/alex/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/resources:/Users/alex/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/alex/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/alex/.m2/repository/weavejester/dependency/0.2.1/dependency-0.2.1.jar
#2019-12-0620:10alexmillerwhat is expected that's different?#2019-12-0620:11alexmillerthat includes the srcDirectory (src)#2019-12-0620:12dominicmOh. So it does. My mistake. Reading long strings is hard. I missed that.#2019-12-0620:23sogaiu(side note) ^ been feeding classpaths to sed to separate them darn things 🙂
also was tipped off to using grep's color option to pick out specific things: https://github.com/jpmonettas/clograms/issues/3#issuecomment-560017206#2019-12-0620:25ghadiI feed them to
clj -Spath | tr ':' '\n'#2019-12-0620:30sogaiunice#2019-12-0620:47dominicmAh, the user is using a version of pack running an older tdeps which predates this jira: https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-96#2019-12-0620:49alexmiller👍#2019-12-0620:52dominicmI need to upgrade to the 0.8.x series really, I'm just not keeping on top of this stuff very well at the moment. Life is busy.#2019-12-0620:56alexmillerwell, should be no breaking changes :)#2019-12-0622:25seancorfield@alexmiller I noticed that recent versions of clojure/t.d.a. actually strip existing newlines from POM files -- was that intentional when you fixed TDEPS-29?#2019-12-0622:26seancorfieldIn my pom.xml files, I tend to have blank lines separating sections at the top-level. clojure -Spom now removes those blank lines. Which is really annoying -- it should not mess with the existing layout.#2019-12-0622:39alexmillerIt was not intentional but makes sense why that would be#2019-12-0622:40alexmillerAs you know, parsing, modifying, then emitting xml is tough to do w/o something like this happening#2019-12-0622:41alexmiller(Really, xslt is a better tool for this job but I didn’t have much luck going down that route)#2019-12-0622:41alexmillerPatches welcome :)#2019-12-0623:02dominicmIs xslt whitespace preserving?#2019-12-0623:41seancorfieldI don't consider it quite annoying enough to create a JIRA issue so I'll just learn to live with it 🙂#2019-12-0623:44alexmillerxslt has a lot of options, not sure. I did look seriously at it back when I wrote the pom sync, but that was a couple yrs ago, don't remember the tradeoffs now.#2019-12-0706:45pyrHi, I know it's not tools.deps per-se#2019-12-0706:46pyrbut related, I discovered pulling private repositories with gitlibs was not possible with HTTP auth#2019-12-0706:46pyrwhen using github tokens for automation tooling this can be an issue for finer grained repo access control#2019-12-0706:46pyrI have the following patch if you think this is worth considering: https://github.com/pyr/tools.gitlibs/commit/6b5c18d37450cd86afb70130a126c242f3fad4b9#2019-12-0709:16alexmillerWe don't want any credentials to show up in deps.edn - with this you'd have plaintext passwords in git urls wouldn't you?#2019-12-0709:22alexmillerwe've had a lot of discussion about https auth (https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-91, https://clojure.atlassian.net/browse/TDEPS-33) and I think the main question to resolve is whether we keep doing jgit or whether we start shelling out to git and I've been trying to get that decision up to the top of my stack (and even briefly did so a couple months ago before I was interrupted).#2019-12-0709:25alexmillerIf we're using git auth, you have a few different choices - I don't think anyone wants to manually type in passwords every time (or any time) and that ruins pretty much any automation workflow, but you can use "store" mode (stored in cleartext file), or "oskeychain" mode on mac, or the "Git Credential Manager for Windows" mode on windows which uses Windows Credential Store, or I think there may be other more generic password oracle options for Git.#2019-12-0709:27alexmillerIf we're using jgit, then it's a matter of where we can read pws from - again you talk to os-dependent things like the keychain, credentials store, etc but we're probably building that ourselves (or maybe something exists, don't know)#2019-12-0709:27alexmillerBut my main constraint is, don't put it in deps.edn.#2019-12-0711:36pyr@alexmiller OK understood. A possible alternative is to go with the NetRCCredentialsProvider. Shelling out to git would solve these issues#2019-12-0711:36pyrI'm using tools.gitlibs here from outside of tdeps which is why having the tokens in the config makes sense to me in this case but I get that for tdeps not necessarily.#2019-12-0711:39pyrI'll drop my commit for now, thanks for the clarification#2019-12-0711:40pyrLooking at the issue, it's indeed shell-out to git or a custom CredentialsProvider implementation which looks for credentials.edn#2019-12-1113:11borkdudeit is possible to generate a classpath without clojure and spec itself on it?#2019-12-1113:12borkdudee.g. here:
$ clojure -Srepro -Sdeps '{:deps {myscript {:git/url "" :sha "89f66ba2101180a98acf88441693840875288ec9"}}}' -Spath
I want only the git dep to be on the classpath#2019-12-1113:12borkdudebut I'm getting:
src:/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/borkdude/.gitlibs/libs/myscript/myscript/89f66ba2101180a98acf88441693840875288ec9:/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
#2019-12-1113:13borkdudeI want to use tools.deps to create a classpath and download deps for a clojure interpreter tool I'm writing#2019-12-1113:14alexmillerVia clj, no (due to the install deps.edn). Via tools.deps, yes.#2019-12-1113:15borkdudeso I'd have to write my own tool that uses tools.deps as a lib? ah#2019-12-1113:15alexmillerYes#2019-12-1113:15alexmillerclj is a Clojure runner#2019-12-1113:15alexmillerIt assumes you’re using Clojure#2019-12-1113:16borkdudeok#2019-12-1113:29borkdudebtw I could see this feature be useful for other environments like self-hosted Clojure / planck, which is already using the clojure like this#2019-12-1113:29borkdudeI'll ask in their channel how they solve this "problem" (more of an optimization really)#2019-12-1113:46alexmillerYou can run tools.deps from clj btw - there is a built in alias for it -A:deps #2019-12-1113:48alexmillerIt’s not exactly a 1 liner but the tools are there (really depends precisely on what you want to do)#2019-12-1114:00borkdudeI'll figure something out. Btw, dirty hack: {:deps {org.clojure/clojure {:local/root "."}}}#2019-12-1114:12alexmillerreally ugly 1-liner:#2019-12-1114:12alexmillerclj -A:deps -e "(println ((requiring-resolve 'clojure.tools.deps.alpha/make-classpath) ((requiring-resolve 'clojure.tools.deps.alpha/resolve-deps) {:deps {'org.ow2.asm/asm {:mvn/version \"7.2\"}} :mvn/repos @(requiring-resolve 'clojure.tools.deps.alpha.util.maven/standard-repos)} nil) nil nil))"#2019-12-1114:12alexmillerprints to stdout /Users/alex/.m2/repository/org/ow2/asm/asm/7.2/asm-7.2.jar#2019-12-1114:13alexmillerasm just used as an example#2019-12-1114:15alexmilleryou could also slurp from deps.edn or whatever (some tools for that stuff in clojure.tools.deps.alpha.reader)#2019-12-1115:10borkdudenice, thanks!#2019-12-1115:36borkdudeI think I'll just use the existing tools though, the entire tooling is already there, caching and all, it'd be a lot of work and error-prone to replicate this. I'll just skip over some known deps that aren't interesting for the interpreter#2019-12-1115:37borkdudeThe idea is that the interpreter is a drop-in replacement for a program you can otherwise run with normal Clojure (give or take some reader conditionals)#2019-12-1115:37borkdudeso re-using most of what's there makes also sense from that perspective#2019-12-1115:38alexmillergotcha - to some degree the :classpath-overrides alias option is a good match for this#2019-12-1115:38alexmilleryou could just override the classpath part for org.clojure/clojure to nothing#2019-12-1115:39alexmillerI guess you'd need to also do clojure's deps though#2019-12-1115:41alexmillerclj -Sdeps '{:aliases {:bye {:classpath-overrides {org.clojure/clojure "" org.clojure/spec.alpha "" org.clojure/core.specs.alpha ""}}}}' -A:bye -Spath#2019-12-1115:42borkdude{:aliases {:babashka {:classpath-overrides {org.clojure/clojure nil
org.clojure/spec.alpha nil
org.clojure/core.specs.alpha nil}}}}
$ clj -C:babashka -Spath
src:::
#2019-12-1115:43borkdudePerfect!#2019-12-1115:50alexmillerI should probably be killing the nils in there :)#2019-12-1115:50alexmillerbut yeaeh#2019-12-1115:54alexmillercommitted that fix for whenever I do next release#2019-12-1115:56borkdudemaybe remove empty?#2019-12-1115:56alexmiller?#2019-12-1115:57borkdudeinstead of remove nil? so empty strings are also "killed"#2019-12-1115:57alexmillerwell, would use str/blank? for that case#2019-12-1115:58alexmillerimo it's bad form to seq-ize strings with empty? vs using a string-specific predicate#2019-12-1115:59borkdudeok#2019-12-1115:59alexmillerdone#2019-12-1116:02borkdudeawesome 🙂#2019-12-1116:12borkdudeNot that I need it right now, but it is possible to override the tools.deps version for the clojure tool to a git dep on master?#2019-12-1116:50alexmillerno, not without hacking the script#2019-12-1116:51alexmillerhacking works fine though, I test stuff that way all the time :)#2019-12-1116:52alexmillerI guess maybe for this use of -A:deps, you might be able to use :override-deps or :classpath-overrides to make that work#2019-12-1116:58alexmillerthat is, there is no hook to affect the classpath of the jvm run that uses tools.deps.alpha to make the classpath, but you could use the existing alias modifiers to change the classpath of the program you're running if it uses tools.deps.alpha#2019-12-1217:00scottloweHi there. I’m working on a project that has a dependency on a large repo of test data, which we don’t want comitted in the main project repo.
Once retrieved, the directory is 295 MB uncompressed on disk. We are seeing OOMEs when tools.deps is retrieving via tools.gitlibs:#2019-12-1217:00scottloweCloning:
#2019-12-1217:00scottloweThis error never occurs locally on macos dev machines, but when running on a Travis CI Linux VM. Ubuntu Bionic.
I note that tools.gitlibs “0.2.64” is using jgit “4.10.0.201712302008-r”.#2019-12-1217:00scottloweDigging a bit futher it looks like jGit is on v5.x series and tools.gitlibs is on the v4.x series. Notice that some recent releases of jGit have memory leak fixes. Hard to say if this relates directly to our issues, though.#2019-12-1217:01scottloweIs this a bug that I should create a ticket for? Or are we using tool.deps outside of its parameters of intended use?#2019-12-1217:02scottloweI’ve tried giving the JVM a good deal of memory, and when that didn’t work, I’ve tried restricting memory to a smaller amount (1.2 GB) so that the JVM can better plan its memory use… but neither approach has worked.#2019-12-1217:03ghadi@scottlowe how specifically did you give the JVM more memory?#2019-12-1217:04scottloweHi Ghadi. I used an env var e.g.: JAVA_OPTS=-Xmx1280m. Less than was available on the Linux instance. I wanted to avoid the Linux OOM-Killer… which may or may not have been related.#2019-12-1217:08ghadiclearing up a couple misconceptions:
1) clj launches two JVMs, one to clone the repos, then another for your project. To modify the JVM for the first one, you need to change this line https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/clojure#L314 in /usr/local/bin/clojure or wherever your script lives
2) JAVA_OPTS isn't a thing, but a convention in some scripts -- not honored by clj#2019-12-1217:09scottloweWow. I didn’t know that. Thanks for the education 🙂#2019-12-1217:09ghadinp#2019-12-1217:10ghaditry bumping up from 256m and your problem may (?) go away -- definitely worth a issue though, right @alexmiller?#2019-12-1217:10scottloweI will go and try some different memory allocations. Could you just confirm that tools.gitlibs should be able to handle this size of dependency, if conditions are right?#2019-12-1217:11ghadinot sure what you want me to confirm 🙂 your analysis of the issue seemed right#2019-12-1217:12ghadibumping JGit is on the radar, as well as maybe making that whole subsystem more resilient#2019-12-1217:13dominicmSeems like loading the code into memory would be a mistake somewhere in the pipeline.#2019-12-1217:13dominicmIt can be written straight to disk.#2019-12-1217:13ghadiyou're making an assumption that it's loading into memory#2019-12-1217:13scottloweYeah. I was wondering about that… should be able to stream any amount of data straight through. Thanks. I’ll go away and experiment, then. Will report back either way… but it may take a few days to be sure that the memory tweaks have worked because we don’t see it every time, because of CI dependency caching etc.#2019-12-1217:14ghadiit might be a large git pack file being decompressed#2019-12-1217:38alexmillerI have some tabs and editors open working on validating the latest versions of jgit, but it's a notch or two down the stack right now. Kind of an interim step towards comparing latest jgit vs shelling out to git for https/ssh access#2019-12-1217:39alexmillergenerally, I'd expect the jvm heap on the cp-making jvm to grow quite a bit larger, but it depends what jvm and version you're using#2019-12-1218:21scottloweAhh. JVM versions... that’s another longer story. In our dev and CI environment we’re currently restricted to using OpenJDK 8 because of a DB dependency. Our Clojure code should be fine running under version 11 and we hope to move to OpenJDK 11 early next year. It’s possible we could use multiple JVM versions in the meantime (to start the DB under a seperate JVM), using a version switcher or something, but that might be asking for trouble as well as more complexity!#2019-12-1218:21scottloweThanks for the comments Alex. I had heard about the shelling out approach for future versions of tools.gitlibs, and I’m hopeful about that approach. I look forward to your findings. :crossed_fingers:#2019-12-1304:48mattyulrichHi all - I’m sure this has been asked before but I just can’t find the answer. Is there a way to signal in my deps.edn that one of my deps is a bom and not a jar?#2019-12-1305:10mattyulrichOr, actually more specifically, I seem to have problem with a bom as transitive deps.#2019-12-1308:13dominicmWhat's a bom?#2019-12-1308:17andy.fingerhuthttps://howtodoinjava.com/maven/maven-bom-bill-of-materials-dependency/#2019-12-1308:56dominicmOops. I was totally expecting it to be a typo of pom.#2019-12-1309:06andy.fingerhutYou may be right, but bom does have a relevant meaning here.#2019-12-1309:17plexusnot to be confused with a byte order mark#2019-12-1314:43mattyulrichHey all - actually noticed some updated support for this. Switched to the latest tools.deps and the transitive dep was resolved.. yeah - I was talking about a “bill of materials” dep..#2019-12-1311:02codeasoneWhat route should I take to get the equivalent of lein uberjar with :aot :all using the Clojure CLI tools?#2019-12-1311:04codeasoneI'm presently using depstar but there doesn't appear to be an equivalent option#2019-12-1317:00seancorfieldWe use depstar at work to build uberjars for production deployment and we do not AOT anything. #2019-12-1400:23codeasoneThanks for replying @U04V70XH6. Along with deploying our uberjar to ECS within a docker image, we also invoke it multiple times within our CI pipeline. My principle motivation was to shave some time off the CI runs, and additionally plain curiosity around why the facility wasn't being supported yet.#2019-12-1400:25seancorfieldFair enough. If you (manually) create a classes folder in your project and add it as an :extra-paths entry when building your uberjar, you can run clojure -e "(compile,'my.entry.point)" as a way to compile the entry namespace (`my.entry.point` here) and its transitive deps.#2019-12-1400:27codeasoneGreat tip: thanks!#2019-12-1400:32seancorfielddepstar tries hard to avoid messing with your project or your classpath: it just packages up whatever you tell it to, using the classpath as constructed by clojure using whatever options you've provided. Because compile requires that classes exists -- or else you can use binding to tell the compiler to put class files somewhere else, that must still exist -- and that the output path of the compiler needs to be on the classpath for depstar to include those files, it just doesn't seem like something depstar itself should try to do.#2019-12-1400:37seancorfieldNote that the depstar README does address this case https://github.com/seancorfield/depstar in terms of Main-Class in the manifest but it stops short of explaining specifically how to compile code and get the classes onto your classpath.#2019-12-1418:18codeasoneThanks for clarifying.#2019-12-1311:54kszaboAOTing is not recommended anymore unless you deeply care about startup perf#2019-12-1311:55kszabobut if you really want to do it, I prefer https://github.com/EwenG/badigeon#2019-12-1312:00borkdudeif you have an :aliases alias in ~/.clojure/deps.edn and one in your project deps.edn, it seems they are not merged?#2019-12-1312:01borkdudee.g. putting:
:babashka {:classpath-overrides {org.clojure/clojure nil
org.clojure/spec.alpha nil
org.clojure/core.specs.alpha nil}}
in ~/.clojure/deps.edn
and in the project.deps::
:babashka {:classpath-overrides {my_gist_script nil}}
does not seem to merge#2019-12-1312:50alexmillerThey merge-with merge at the aliases level iirc#2019-12-1314:53mattyulrichI’m sure this has been asked before, but can’t seem to find it. When within a running clj repl, is it possible to resolve new deps; kind of like boot-clj set-env! ?#2019-12-1315:13alexmillerthere's a speculative feature called add-lib in a branch for this#2019-12-1315:13alexmillerhttp://insideclojure.org/2018/05/04/add-lib/#2019-12-1315:13alexmiller(sha in that article is out of date)#2019-12-1919:35vinnyataidewhere do I find the correct sha?#2019-12-1919:47alexmillerwhatever the newest sha in the add-lib branch is generally https://github.com/clojure/tools.deps.alpha/tree/add-lib#2019-12-1919:50alexmillerit was pretty far out of date so I just merged it in 19d197ab221d37db750423eb970880cb87a91100#2019-12-1920:08vinnyataidethanks#2019-12-1315:21mattyulrichWow - this is great! Thanks Alex!#2019-12-1315:24alexmillerI think we will probably add this, but some of the details are likely to change#2019-12-1317:01seancorfield@codeasone why do you want to AOT?#2019-12-1317:02alexmillerit is perfectly reasonable to AOT a final uberjar for deployment#2019-12-1318:48hlolliI asked this question few months ago, but since that time, did anyone write a lein-ancient equivalent for tools.deps?#2019-12-1318:48hlolliI asked this question few months ago, but since that time, did anyone write a lein-ancient equivalent for tools.deps?#2019-12-1318:55Laverne SchrockThere's depot and deps-ancient (I've only used depot). https://github.com/clojure/tools.deps.alpha/wiki/Tools#deps-management#2019-12-1318:58seancorfieldWe've been using Depot very happily (although I've recently developed a bash script that does crazy stuff with clojure -Stree and a synthesized deps.edn file to get outdated deps in a way that suits us better)#2019-12-1319:16hlolliwow nice, essential tool in any toolkit. I knew someone had made it by now 🙂#2019-12-1319:16hlollithanks#2019-12-1319:19borkdude❤️ depot#2019-12-1319:20borkdude@U04V70XH6 care to share the bash script? I'd be curious if I can port it to babashka and make a nice example out of it 🙂#2019-12-1319:22seancorfieldTrust me when I say it's "crazy stuff" -- and it relies very heavily on our custom build shell script that drives our use of clojure...#2019-12-1319:22borkdudeok#2019-12-1319:25seancorfieldThe TL;DR is that it runs clojure -Stree to a file, runs that through sed etc to produce a temporary deps.edn with "RELEASE" for all of the dependency versions (that were :mvn/version explicit versions), then runs clojure -Stree on that, and then it compares the output of both trees, with some fudging to account for only the top two levels of dependencies...#2019-12-1319:26borkdudesounds like a good hack to me 🙂#2019-12-1319:33seancorfieldRunning Depot reliably on our huge monorepo and getting all the outdated deps that matter and avoiding the noise from transitive deps that I don't care about was very tricky, especially since we use CLJ_CONFIG to setup a baseline deps.edn with :override-deps for a lot of stuff...#2019-12-1319:36seancorfield...any t.d.a. tooling that tries to read and process deps.edn files in isolation can run into problems with our setup -- tooling that uses the same process to read/merge the environment as clojure itself does works better for us.#2019-12-1320:00borkdude@U04V70XH6 https://github.com/borkdude/babashka#list-outdated-maven-dependencies
Thanks for the idea 🙂#2019-12-1320:04seancorfieldWow, hahaha!! Important to note that your code won't work with any deps.edn that relies on ~/.clojure/deps.edn (or CLJ_CONFIG) to override/default any deps... 🙂#2019-12-1320:05borkdudeyeah, this is just a quick sketch which should be tuned to the needs of the scripter#2019-12-1320:06seancorfieldIt just goes to show what can easily be done -- and it would work for "most people" I suspect.#2019-12-1320:06borkdudeprobably one could come up with a way to use git to inspect newer shas as well#2019-12-1320:07borkdudebut that's an exercise for the reader 🙂#2019-12-1320:11borkdudeOne could even use this for project.clj by just parsing the deps from it, generate a deps.edn and then run this script#2019-12-1321:16scottlowe@ghadi I’m reporting back on yesterday’s tools.gitlibs OOME issue, which was seen when downloading large repo dependencies: bumping JVM Xms from 256m to 512m seems to have worked. I’ve not seen a failure yet, anyway. Thanks again for your instantaneous help.#2019-12-1321:16ghadiNo problem. What jvm version and vendor do you use btw?#2019-12-1321:18scottloweFor the CI environment it’s OpenJDK 1.8.0_222#2019-12-1321:28alexmillerseems pretty weird that jvm wouldn't grow to 512m#2019-12-1321:28alexmillerI wonder if you're getting client jvm by default instead of server#2019-12-1321:31scottloweIndeed - I had wondered myself why this would be so; I shouldn’t need to babysit the jvm (?); but it does seem to be the server jvm:#2019-12-1321:31scottlowe$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
#2019-12-1321:32alexmillerusing same version locally, I get an initial of 256m and max of 4g with no settings#2019-12-1321:33alexmillerwith -Xms256m as set in the script, I see the same#2019-12-1321:35alexmillerhowever, the max is dependent on the available ram - I think it's 1/4 total ram by default#2019-12-1321:35scottloweI agree that it is odd. This is on a Travis CI instance: Ubuntu 16.04.6 LTS (xenial). We hope to move to more recent versions of the JVM and Ubuntu soon. We’ve never seen this issue on developer machines (macos). The CI environment is more memory constrained than our dev machines. 7.5 GB, I think; but some of that memory is taken up by docker processes, as well as a JVM database process.#2019-12-1321:38andy.fingerhutMaybe the Travis CI instance has 1G RAM default?#2019-12-1321:40alexmillercertainly could be something like that#2019-12-1321:47scottloweHave checked the Travis CI instance: System Memory: 7680MiB, 64 bits#2019-12-1823:29caiois there an equivalent of lein's :java-source-paths for deps.edn?#2019-12-1823:32alexmillerno, clj doesn't support compilation or java source#2019-12-1823:33alexmillerit's intended for Clojure source only projects that don't require "building"#2019-12-1908:45vlaaadand yet there are tools already on top of tools-deps that do building and stuff#2019-12-1908:47vlaaadalthough they are mostly about making jars from clojure source, haven't seen any that compile java#2019-12-1909:04stijnis there a limit on the length of paths for tools.deps? when running clj (it's datomic ions push, but I don't think that matters) in Codebuild with local cache enabled for gitlibs it will fail with the following exeption
{:command-failed
"{:op :push :region us-east-1}",
:causes
({:message "Unable to transform path",
:class ExceptionInfo,
:data {:home "/root",
:prefix "/root/.gitlibs/libs",
:resolved-coord
{:git/url "<redacted>",
:sha "e7cc7c48ea0af52f9380c61d2d946cac5d7068e0",
:deps/manifest :deps,
:deps/root "/codebuild/local-cache/custom/b11f68d510f24e13d8132571e19ad44f6bd772898be1fc7b98e76106d7e1ab50/root/.gitlibs/libs/<redacted>/<repo>/e7cc7c48ea0af52f9380c61d2d946cac5d7068e0",
:paths ["/codebuild/local-cache/custom/b11f68d510f24e13d8132571e19ad44f6bd772898be1fc7b98e76106d7e1ab50/root/.gitlibs/libs/<redacted>/<repo>/e7cc7c48ea0af52f9380c61d2d946cac5d7068e0/src"]}}})}
If I run locally I don't see this problem and also when turning off cache for gitlibs, the problem disappears. Codebuild cache will create a symlink of /root/.gitlibs to /codebuild/local-cache/custom/b11f68d510f24e13d8132571e19ad44f6bd772898be1fc7b98e76106d7e1ab50/root/.gitlibs making the directory maybe too long?#2019-12-1913:39alexmillerI’m not aware of a length issue, should probably ask in #datomic#2019-12-1913:14sogaiu@vlaaad i think badigeon compiles java: https://github.com/EwenG/badigeon#2019-12-1913:20vlaaadoh, right!#2019-12-2118:56practicalliIs it possible to create a project structure as follows with clj-new or any similar tools. Is it just a matter of creating my own template? Or just hacking the project files once its created
advent-of-clojure
|_ src
|_ practicalli
|_ advent_of_code_2019
|_ 01_trinity_of_rocket_equasion.clj
It would have the matching path for test branch too.
/cc @seancorfield#2019-12-2119:06seancorfield@jr0cket This will get you pretty close to what you want:
clj -A:new app practicalli/advent-of-code-2019.01-trinity-of-rocket-equation
#2019-12-2119:06seancorfield(! 951)-> tree advent-of-code-2019.01-trinity-of-rocket-equation/
|____
| |____.gitignore
| |____.hgignore
| |____CHANGELOG.md
| |____deps.edn
| |____doc
| | |____intro.md
| |____LICENSE
| |____README.md
| |____resources
| | |____.keep
| |____src
| | |____practicalli
| | | |____advent_of_code_2019
| | | | |____01_trinity_of_rocket_equation.clj
| |____test
| | |____practicalli
| | | |____advent_of_code_2019
| | | | |____01_trinity_of_rocket_equation_test.clj
#2019-12-2119:17practicalliThat great thanks. All I would do is just edit the top level directory name and its perfect.
I was looking at the clj-new documentation and thinking I could also do this with generators. adding new namespaces with clj -A:new -m clj-new.generate ns=practicalli.advent_of_code_2019.01_,,,.clj . This would be handy to script up the adding of a namespace for each day, especially if I wanted a separate -data.clj file for my unique data (keeping the solution namespace cleaner).#2019-12-2119:30seancorfieldYup, once you have the project folder, you can add new namespaces via the generator logic.#2019-12-2512:59borkdudemade a thing: https://github.com/borkdude/deps.clj#2019-12-2518:44jApolgies if this is posted in the wrong group! I created a uberjar using clj -A:new app and clj -A:uberjar myproject.jar alias from @seancorfield’s dot-clojure aliases. I'm getting the following error message though: Error: Invalid or corrupt jarfile myproject.jar
I copied the aliases from the deps.edn from dot-clojure over to the deps.edn in the project to run the :uberjar alias. How do I make a working uberjar?#2019-12-2519:13seancorfield@watchtheblur The README on depstar explains all that...#2019-12-2519:13seancorfieldhttps://github.com/seancorfield/depstar#2019-12-2519:14seancorfieldAt work we run all our uberjars like this: java -cp /path/to/the.jar clojure.main -m entry.point -- which will run entry.point/-main as the "main" function. But the readme explains several other approaches.#2019-12-2519:16jIt worked! Thanks for the assist again!#2019-12-2607:09seancorfieldseancorfield/clj-new {:mvn/version "0.8.0"} -- https://github.com/seancorfield/clj-new/releases/tag/v0.8.0 -- adds new options and pom.xml generation (for library and template projects) -- Happy Holidays! 🙂#2019-12-2614:49rafaelHi. I'm wondering if there are best practices for locally referencing (or referencing via git) a project that relies on gen-class ? If I explicitly compile, everything works out fine, and it was easy to build a task to do that. I'm not sure how to fit it into the development workflow, though. Should I require every contributor to compile locally before working on dependent projects or should I just commit the class files to git and be done with in? Are there alternatives? Can I add a hook somewhere that will compile the gen-class project before starting a repl on a project that depends on it?#2020-12-3013:49mynomotoWe use make to ensure that the necessary classes are compiled when working on dependent projects.#2019-12-2810:09sveriHi. I am using cursive which can start an nrepl with deps. Now, when the repl is started I want to switch to a specific namespace and run a function inside.
Using the clojure.main repl I was able to do that by providing a main opts entry point like this:
:aliases {:repl {:main-opts ["-e" "((requiring-resolve 'de.sveri.sg.user/startup))"]}}
Is there something similar for the nrepl?#2020-12-3000:16seancorfieldseancorfield/clj-new {:mvn/version "0.8.1"} -- https://github.com/seancorfield/clj-new/releases/tag/v0.8.1 -- adds pom.xml to app template as well; adds jar/uberjar building aliases to all three built-in templates; expands documentation on the built-in templates how to modify pom.xml generation via the -e / --env options.#2020-12-3000:19seancorfieldIn particular, note that projects generated from the app template now let you run clojure -A:uberjar to produce a JAR file that can be run with java -jar -- the :uberjar alias includes a compile call on the main namespace and the classes folder is added to the paths when building the JAR.#2020-12-3009:49felipebarrosI would like to mimic the effect of npm install or lein deps with tools-deps. That is, to fetch the dependencies and exit. Running clj -R:alias always leaves me in a running REPL and I could not find an option in the documentation to do what I want.#2020-12-3009:54delaguardoclj -Stree will fetch all dependencies, print out complete deps tree and exit#2020-12-3009:55felipebarrosThanks, that will work :)#2020-12-3010:24borkdudeanything with -Sforce works#2020-12-3010:25borkdudee.g. -Sforce -Spath also works#2020-12-3014:32alexmillerThese are good answers. you can also clj -e nil#2020-12-3014:32alexmillerif you want to run something and immediately exit#2020-12-3016:46felipebarrosThank you :)#2020-12-3101:57fabraoHello all, I´m confuse about using boot or deps.edn. What is the advantage of using one beside other? I switched from lein to boot because seancorfield told me that with boot we can load deps "on-the-fly" and I was using a lib in lein to do this. I saw this https://twitter.com/seancorfield/status/1171275237805060096?s=20 and https://www.youtube.com/watch?v=CWjUccpFvrg&t=46s and in terms of dev workflow it seems other way to do things. What is the most benefit of using deps.edn?#2020-12-3102:06seancorfield@fabrao Boot provides a whole bunch of tools "out of the box" -- although it does not incorporate a test runner (last I checked you needed to add Adzerk's boot-test for that, nor is "boot new" technically built-in but it's an easy command-line dependency to use). As noted, the boot repl allows you to add new dependencies dynamically, and it has some file management tooling built in (such as building jar and uberjar files).#2020-12-3102:07seancorfieldThe CLI/`deps.edn` is simple and lightweight, with a range of easy to add tooling for running tests, building jar files, etc. There's an add-lib branch of tools.deps.alpha which you can use -- illustrated in my dot-clojure file -- to add new dependencies dynamically.#2020-12-3102:08seancorfieldI would classify lein and boot as "easy" -- lein more so than boot -- whereas I would classify clj as "simple" (in the general Clojure sense of simple vs easy).#2020-12-3102:09seancorfieldclj is about small, composable tools -- very much in Clojure's "idiomatic world view".#2020-12-3102:12seancorfieldWe switched from Leiningen to Boot back in 2015 because we needed more/easier programmability in our build chain. We developed a lot of custom tasks once we switched to Boot -- thousands of lines of dev/test/build code. We kind of lost sight of simplicity there somewhere 🙂#2020-12-3102:14seancorfieldOur Boot setup already had dependencies in EDN files -- albeit a different format to deps.edn -- so when we were getting ready to switch, I wrote boot-tools-deps (since archived, as a bad idea), as migration step, but in the end we didn't use it.#2020-12-3102:16seancorfieldAs part of our switch to deps.edn, we took a long, hard view at all our Boot tasks and decided to streamline/simplify things. We're very happy with the decision to change over. We have just a simple shell script that wraps clojure so we can run multiple CLI commands from a single invocation.#2020-12-3102:17seancorfield(I will note that we were running into a couple of issues with Boot due to the size of our codebase and our heavy use of "pods", Boot's isolation feature, so we were "pushed" a bit -- it wasn't entirely a "jump")#2020-12-3102:22seancorfieldHappy to chat more via DM any time about tooling, if you want @fabrao (it looks late for you right now -- you're about six hours ahead of me).#2020-12-3102:31fabraowell, I don´t want to bother you at this time, but any time for me#2020-12-3102:34fabraoThe subject about deps.edn started in Clojure Brazil Telegram Group and your twit shows up and I research about it#2020-12-3102:41seancorfieldIt's early evening for me -- feel free 🙂 I'm online about 9am to 11pm Pacific most days 🙂#2020-12-3102:44fabraoHello Sean, how are you? I saw you using REBL instead REPL#2020-12-3117:23kennyDoes anyone recall the solution to "UnknownHostKey: https://github.com. RSA key fingerprint is ...."? We've started getting this on CircleCI. I'm fairly certain I've seen it before and can't remember the solution. The CI has a checkout SSH key.#2020-12-3117:41kennyTurns out our CI image didn't have ssh or git installed. Interestingly, CircleCI still had some way of getting the repo onto the machine without either of those. Adding them fixed this issue.#2020-12-3120:23ghadiThe CI image doesn’t need git or ssh#2020-12-3120:23ghadiBut it does need the github identity key in the .ssh/known_hosts#2020-12-3120:23kennyWell in this case it does 🙂 CircleCI adds the checkout ssh key only if ssh is installed. Because ssh wasn't installed, the key was not added. The CLI therefore could not clone the repo.#2020-12-3120:24ghadiI’d “pin” these in your circleCI yml file#2020-12-3120:24ghadik#2020-12-3119:39seancorfieldI finally caved and added an AOT option for uberjars to seancorfield/depstar {:mvn/version "0.4.1"} -- https://github.com/seancorfield/depstar -- there are some caveats: you need to create the classes folder yourself (as a one-off) and you need to ensure that folder is on the classpath when you actually run depstar (the README suggests a :uberjar alias that takes care of that). It only AOTs the -m / --main namespace (but that transitively compiles all namespaces that are directly referenced from it -- it is not "compile all", like Leiningen).#2020-12-3119:42seancorfieldI also updated seancorfield/clj-new {:mvn/version "0.8.2"} -- https://github.com/seancorfield/clj-new -- to take advantage of that for the app template; this also fixes the group/artifact in the pom.xml for generated template projects.#2020-12-3119:53seancorfield> clojure -A:new app myname/myapp
Generating a project called myapp based on the 'app' template.
> cd myapp/
> clojure -A:uberjar
Compiling myname.myapp ...
Building uber jar: myapp.jar
Processing pom.xml for {myname/myapp {:mvn/version "0.1.0-SNAPSHOT"}}
> java -jar myapp.jar
Hello, World!
> #2020-12-3120:01seancorfield(for lib and template types you can clojure -A:jar to build a JAR that should be deployable to http://clojars.org pretty much as-is, with a near-ready-to-roll pom.xml file)#2020-01-0208:56robert-stuttafordprobably an FAQ but my google-fu is lacking ; is there an order of precedence among :mvn/version and :local/root? could i somehow use both, where local is used for dev, but mvn is used for ci? has anything been written about producing this sort of environment specific variance?#2020-01-0209:27delaguardoWe are using special alias for CI builds with :override-deps key inside#2020-01-0211:07dominicmIt'll be an array map, so whichever namespace is found first most likely. As mentioned you should use override deps.#2020-01-0214:09alexmillerthe way the code is written (https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions.clj#L32-L37) it depends on the first namespace key found in a map, so you should not assume any arbitrary order. Really, I'd say this is an error.#2020-01-0214:19alexmillerthe whole point of stuff like :override-deps is to explicitly support the use case of changing a dep source#2020-01-0306:35robert-stuttafordthanks folks!#2020-01-0306:39robert-stuttafordWorks beautifully.#2020-01-0211:00dharriganDoes this -Sdeps DEPS Deps data to use as the final deps file take a map?#2020-01-0211:01dharrigani.e., if I want to pass in mvn/repo configuration on the command line, thus ignoring whatever is set in any deps.edn file, is it this switch I require?#2020-01-0211:08dharriganyup is the answer 🙂#2020-01-0211:16dharriganWhere does deps store downloaded deps, i.e., jars? In the .m2 directory?#2020-01-0213:20borkdude@dharrigan you can check that with -Spath#2020-01-0213:20dharriganah, thanks#2020-01-0213:21borkdudeThis is an example for mvn + git:
src:/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/borkdude/.gitlibs/libs/borkdude/medley/91adfb5da33f8d23f75f0894da1defe567a625c0/src:/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
#2020-01-0213:21dharriganyup#2020-01-0213:21dharriganthanks#2020-01-0217:49seancorfieldseancorfield/depstar {:mvn/version "0.5.0"} now manages the AOT compilation folder automatically -- no need to manually create classes or add it to the classpath; seancorfield/clj-new {:mvn/version "0.8.3"} updated to use latest depstar (so app template no longer needs classes folder).#2020-01-0221:06vlaaadAnd I am still on 0.2.1 because it does what it needs to do 🙂#2020-01-0221:28seancorfieldThat's good to hear! 👍:skin-tone-2:#2020-01-0221:29seancorfield0.5.1 and 0.8.4 just released to add support for projects where the main namespace has - in its path and/or its name.#2020-01-0703:57Nate Suttonhow do you run clojure.test tests with a deps.edn file?#2020-01-0703:58Nate SuttonI feel like I must be missing something#2020-01-0704:01Nate Suttonoh, it doesn't come with a test runner#2020-01-0704:01Nate Suttonthat'll do it#2020-01-0704:23andy.fingerhutHere is an alias I added to the deps.edn of the core.rrb-vector library: https://github.com/clojure/core.rrb-vector/blob/master/deps.edn#L30#2020-01-0704:24andy.fingerhutI wouldn't be surprised if there are better ways, but that is effective and straightforward#2020-01-0704:25Nate Suttoncould I ask by all the commas instead of spaces?#2020-01-0704:25Nate SuttonI understand that they're considered whitespace in clojure#2020-01-0704:25Nate Suttonjust wondering if that's some kind of limitation of :main-opts or something#2020-01-0704:31seancorfield@nate_clojurians Have you looked at https://github.com/seancorfield/dot-clojure ? That has a lot of stuff that is very helpful when you're getting started with CLI/`deps.edn`#2020-01-0704:31potetmgonna guess the commas are because both ' and " are special in the shell#2020-01-0704:31potetmso , is a “guaranteed neutral space”#2020-01-0704:32Nate Suttonbut why is space an issue then?#2020-01-0704:32seancorfieldThe main opts are round-tripped from shell clojure into files on disk and then back again into the shell and then to java.#2020-01-0704:33potetmIt’s usually safe to assume these kind of shenanigans when storing shell commands. Nobody does it right.#2020-01-0704:32seancorfieldSo the quoting/space combinations break in that round-tripping.#2020-01-0704:32Nate Sutton@seancorfield I did see that but I thought it incomplete because it didn't have a bare example of just using clojure.test itself. turns out there's no test runner#2020-01-0704:33seancorfieldCognitect's test-runner is in there.#2020-01-0704:33seancorfieldHence the :test alias and the :runner alias.#2020-01-0704:33Nate Suttonyep, and I'm using that now. I didn't know there wasn't a built-in test runner in clojure and I was avoiding the additional dependency#2020-01-0704:34seancorfieldWell, you can run tests without the test-runner -- clojure.test does have some basic test running logic -- but you won't have a very nice experience 🙂#2020-01-0704:35seancorfieldclojure.test/run-all-tests will run all tests that have been loaded (in require'd code).#2020-01-0704:36Nate Suttonthat seems like the part I was missing, because I did try to use clojure.test/run-all-tests as the main function#2020-01-0704:36Nate Suttonwell, that and other pieces#2020-01-0704:36Nate SuttonI wish there was a note about tests on the deps + cli page on http://clojure.org#2020-01-0704:46alexmillerif I could trouble you to file an issue so I remember, I'd be happy to do so https://github.com/clojure/clojure-site/issues#2020-01-0704:47Nate Suttonawesome, thanks!#2020-01-0704:50Nate Suttonwoo https://github.com/clojure/clojure-site/issues/444#2020-01-0704:37seancorfieldAll the test runners out there look for "test" files (by various means), load the code, and then run all the tests, under whatever filters you provide.#2020-01-0704:38Nate Suttonfor sure, I was just surprised there wasn't one built in, or a note somewhere about there not being one and that you'll have to pick a test runner#2020-01-0704:38Nate Suttonmaybe there is a note somewhere and I didn't find it#2020-01-0704:38Nate Suttontotally possible ¯\(ツ)/¯#2020-01-0704:39seancorfieldFWIW, Boot doesn't have a test runner built-in either. Only Leiningen does.#2020-01-0704:40Nate Suttonahhh ok, I started with lein#2020-01-0704:40Nate Suttonstill, I imagine it would have been confusing for me even if I hadn't#2020-01-0704:40seancorfieldMost of the test runners out there are additional dependencies that offer more features than Leiningen's built-in stuff...#2020-01-0704:42seancorfield@nate_clojurians This will probably be helpful for you https://cljdoc.org/d/expectations/clojure-test/1.2.1/doc/getting-started#running-tests#2020-01-0704:43Nate Suttonisn't this for a third party tool?#2020-01-0704:43seancorfieldIt's under consideration to spin off clojure.test into a Contrib lib for Clojure 1.11. I've volunteered to maintain it and one of the things I'll do is add a bunch of documentation to the project, that covers how to run tests with a variety of tools, and how to filter tests etc.#2020-01-0704:43Nate Suttonoh, cool#2020-01-0704:44Nate Suttonwell, I have tests running now, so that's nice#2020-01-0704:44seancorfieldYes, it's for the clojure.test-compatible version of Expectations that I maintain -- but a lot of what's at that link would become "canon" documentation in clojure.test, if I get to maintain it.#2020-01-0704:51Nate Suttonthat looks good#2020-01-0704:51Nate Suttonthe with-test thing is new to me and that'll be super handy#2020-01-0704:51Nate Suttonthanks, sean#2020-01-0704:59seancorfieldwith-test is tricky to use because the default setup for all the test runners will not see those tests -- hence my notes about what needs to be done 🙂#2020-01-0705:06Nate Suttonin this case it's in the context of babashka, where a single file with all deps and tests is desirable#2020-01-0705:07Nate Suttonahhh spartan.test doesn't seem to include a with-test macro#2020-01-0705:07Nate SuttonI'm sure it could be added#2020-01-0808:55plexus@nate_clojurians you might want to look into Kaocha. It was conceived as a test runner for deps.edn that's less spartan than the cognitect runner#2020-01-0810:30Nate Suttonthanks, I'll check it out :)#2020-01-0816:36practicalliIf your editor has a test runner, eg. CIDER for Emacs, then you only need to ensure the 'test' directory is included in the path. You can add it to :path or as an alias.#2020-01-0816:37practicalliI tend to switch betweeen cognitect-labs test runner and eftest (weavjester). I also am trying out Kaocha as its very comprehensive.
I have test-runner aliases in my $HOME/.clojure/deps.edn file
https://github.com/practicalli/deps-edn-examples/blob/master/deps.edn#L114-L137#2020-01-0816:40Nate SuttonI do use cider but I'm also running tests in babashka and don't want to work out using a socket repl instead of nrepl with emacs#2020-01-0817:21plexusCan you elaborate a bit on how you run tests in babashka? What would you expect from a test runner in that environment?#2020-01-0817:28borkdudeWe can take this to #babashka#2020-01-0817:28borkdudeBit of context: this is an intermediate lib for running tests until bb gets clojure.test:
https://github.com/borkdude/spartan.test#2020-01-1010:15Mehdi H.Hi all! I have some troubles removing SLF4J multiple bindings in my project. I know it's been debated here before (https://clojurians-log.clojureverse.org/beginners/2016-10-21 or even in this channel : https://clojurians-log.clojureverse.org/tools-deps/2018-10-12), but at the time it seems that you could find it easily only if you were using lein (I am on CLI tools). Has anything changed since then? That would help me a lot! Thanks!#2020-01-1010:50Mehdi H.Ok never mind, clj -Stree does the trick 🙂#2020-01-1307:34slipsetI ran into a subtle problem over the weekend with clj -Spom on https://github.com/clj-commons/pomegranate. The original pom.xml has the dep for wagon-provider-api here https://github.com/clj-commons/pomegranate/blob/master/pom.xml#L101 whereas the pom.xml generated by clj -Spom with this deps.edn https://github.com/clj-commons/pomegranate/blob/master/deps.edn#L12 put the dependency declaration for wagon-provider-api earlier in the pom.xml.#2020-01-1307:34slipsetOne reaction to this would of course be :shrug: , but from https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html we have that#2020-01-1307:34slipset> Note that if two dependency versions are at the same depth in the dependency tree, the first declaration wins.#2020-01-1307:36slipsetAnd the kicker is that wagon-provider-api depends on an older version of plexus-util than the other deps of pomegranate, so by changing the order of the deps in the pom.xml, pomegranate broke.#2020-01-1307:38slipsetThinking of it, I’m not sure why I’m writing this, I guess it’s just as an amusing story.#2020-01-1310:48slipsetI guess there might be a question in here: Is the order in which the deps are listed in the generated pom.xml predictable in any way?#2020-01-1310:55dominicmI wouldn't think so. It operates on a hashmap.#2020-01-1314:01alexmillerno, it's not predictable#2020-01-1314:02alexmilleryou could exclude the plexus-util in whatever branch and then it wouldn't matter, right?#2020-01-1314:35slipsetThat's correct, and what I ended up doing.#2020-01-1317:03aviDoes tools.deps.graph support gitlibs?#2020-01-1317:03alexmillerYes#2020-01-1317:06aviHmm, I see, it looks like maybe not when --size is specified?#2020-01-1317:06aviwhich makes sense, but perhaps it would be good to display an error message rather than a stacktrace…#2020-01-1317:06avi/Users/avi.flax/.gitlibs/libs/expound/expound/9b5778a1a4ed91e2090308a6648ee9072076925a is not a relative path
java.lang.IllegalArgumentException: /Users/avi.flax/.gitlibs/libs/expound/expound/9b5778a1a4ed91e2090308a6648ee9072076925a is not a relative path
at $as_relative_path.invokeStatic(io.clj:414)
at $file.invokeStatic(io.clj:426)
at $file.invoke(io.clj:418)
at clojure.tools.deps.alpha.extensions.git$eval1027$fn__1029.invoke(git.clj:43)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.graph$get_size.invokeStatic(graph.clj:95)
at clojure.tools.deps.graph$get_size.invoke(graph.clj:93)
at clojure.tools.deps.graph$make_dep_node.invokeStatic(graph.clj:107)
at clojure.tools.deps.graph$make_dep_node.invoke(graph.clj:99)
at clojure.tools.deps.graph$make_graph$fn__1522.invoke(graph.clj:125)
#2020-01-1317:09alexmillerThat’s possible, I thought I tested that, but maybe not. It’s supposed to fall back to 0 in that case#2020-01-1317:10aviWould you like a bug report somewhere?#2020-01-1317:10alexmillerI’ll take a look when I get a chance#2020-01-1317:42aviBTW when you do take a look at this, I’m thinking maybe using 0 for gitlibs is a little misleading? It’s more of an unknown… perhaps something like -1 or “unknown” or :man-shrugging: or just leaving it blank…?#2020-01-1317:10alexmillerNah#2020-01-1317:10avi🆗 thanks2#2020-01-1317:10aviBTW reviewing the deps graph for my project is fascinating, and very informative — thank you!#2020-01-1317:10avi#2020-01-1317:11alexmillerSorry size isn’t working!#2020-01-1317:13aviNo big deal!#2020-01-1317:14aviIt’s already immediately obvious that I have a single dep bloating my project. It’s a dep I’ve been planning to drop eventually, so now I’m even more motivated to so do!#2020-01-1317:15aviHere I’ve manually highlighted that dep and all its deps — it’s pretty dramatic!#2020-01-1317:51alexmillernice - I'd like to also have an output mode to export the dot file itself instead of the image so you can further mess with it#2020-01-1317:51avi:+1:#2020-01-1317:51aviInterested in PRs?#2020-01-1317:52alexmillernah, it's part of other changes I want to make#2020-01-1317:53avi🤔 it’d be cool if there was a feature to assign a unique background color to each branch from the root — might make it easier to visualize the impact of each root-level dep#2020-01-1318:00alexmilleryou could drop that in a question at https://ask.clojure.org if you want, category of "Contrib libs / tools.deps.graph" and label "request"#2020-01-1318:01alexmillerworth playing with. I experimented with a variety of color things, never found something that I thought helped, but I didn't try that!#2020-01-1318:03aviCool, will do!#2020-01-1320:21avihttps://ask.clojure.org/index.php/9028/suggestion-color-coding-dependency-subtrees#2020-01-1321:27seancorfield@alexmiller A question about tools.deps.graph: it seems that it reads just the current deps file so if you have overrides provided in another deps file, it won't see them? At work we have a "user-level" deps file in the monorepo that "pins" versions of everything via a :defaults alias and then we have {} for deps in each project deps.edn file.#2020-01-1321:27seancorfieldCan I specify multiple deps.edn files to read/merge on the command-line?#2020-01-1321:29seancorfield(so far everything I've tried in the context of our work repo has either failed outright or produces a blank canvas claiming the size is -1 x -1)#2020-01-1321:32seancorfieldIf I generate to a file, I seem to get a graph as expected, so it's perhaps only the default render to a graphics window that fails...#2020-01-1321:34alexmillerit's the same as clj#2020-01-1321:34alexmillerin that it uses mostly the identical code#2020-01-1321:34alexmillerso, no you can't use multiple deps.edn files#2020-01-1321:35seancorfieldAh, I was confused by the results I was seeing... it does take the user-level deps into account after all?#2020-01-1321:35alexmillerit should#2020-01-1321:35alexmillerunless I'm misremembering that#2020-01-1321:36alexmilleryeah, it does#2020-01-1321:38seancorfieldbuild [ graph api -a :defaults -o /tmp/graph.png ]
which ends up being
cd api; CLJ_CONFIG=../versions clojure -A:graph:defaults -a :defaults -o /tmp/graph.png
does indeed produce the correct graph for our api subproject. I think getting the blank popup was throwing me off there.#2020-01-1416:17rschmuklerHey all! I'm consuming a library that seems to have packaged itself as an uberjar including all of its dependencies. I also depend on one of the libraries it uses and it looks like that it's overwriting the requires to an old version (which requires a java class that isn't packaged in the new one). Is there any way for me to force my project to use the new version instead of what it's using?#2020-01-1416:18rschmuklerI'm not super familiar with the internals of how this all works, so if the above diagnosis is complete nonsense, please excuse me!#2020-01-1416:21alexmillermakes total sense#2020-01-1416:21alexmillerand no, not easily#2020-01-1416:21alexmilleryou should tell that lib to stop doing that :)#2020-01-1416:23alexmillerthe effect you need is to force your lib version earlier in the classpath ordering. tools.deps mostly does not provide support for explicit ordering#2020-01-1416:25alexmillerI possibly could come up with a hacky workaround, but it would be a hack. the real solution is the lib#2020-01-1703:30alexmillertools.deps.alpha 0.8.264 and clj 1.10.1.496 are now available.#2020-01-1703:30alexmillerThe main change in this is that support for s3 maven repos has been completely replaced and is now using a new impl of the Maven resolver transporter API based on the cognitect aws-api. The old version used the transporter-wagon adapter, the unmaintained spring s3 wagon, and the AWS Java SDK. Those deps have been happily dropped on the floor, and all other deps were bumped as well. Support for some newer AWS credential sources may be better, particularly when running on AWS. Also, you can explicitly set the s3 bucket region in your deps.edn like {:mvn/repos {"my-bucket-repo" {:url ""}}} - this is a feature of this transporter, not a general AWS thing.#2020-01-1703:31alexmillerThe intention here is that any existing use of an s3 maven repo should continue to work as it did before. If you find that not to be the case, please let me know! It has been tested in a number of configurations by several people, but you know how it goes.#2020-01-1703:33alexmillertools.deps tool makers may want to do a quick check as this version has some significant transitive dependency differences (hopefully mostly for the better!)#2020-01-1703:38alexmillerdocs have not yet been updated, but I'll do that tomorrow#2020-01-1704:45tianshuHow can I get my clojure cli version?
I usually install with system package manager, It only shows the version of Clojure.#2020-01-1707:58Michaël SalihiTo get the clojure cli verison without launching the REPL as clj -Sverbose do, you can also use clj -Sdescribe.
@alexmillerRight ?#2020-01-1705:31alexmillerclj -Sverbose#2020-01-1705:33tianshu@alexmiller thanks!#2020-01-1707:07slipsetI was wondering about the version thing the other day as well, and tried to invoke clj --version which printed the help-text, which I failed to parse (ie I didn’t understand that Print important path info to console meant “and also print version”#2020-01-1707:07slipsetWould it be a nice addition to print the version string in the help-text as well?#2020-01-1707:21slipsetlooking through clojure/brew-install this should be simple enough, but is there a Jira project for this project where one might submit a patch?#2020-01-1714:33alexmilleryou can use TDEPS for tickets, but I just added it to the repo#2020-01-1715:38slipsetThanks 🙂#2020-01-1715:38slipsetHave a great weekend!#2020-01-1715:54alexmillerthx, you too#2020-01-1709:32borkdudedeps.clj updated to 1.10.496: https://github.com/borkdude/deps.clj/releases/tag/v0.0.6#2020-01-1718:28rschmuklerHey all! I'm looking to publish a library on clojars with a deps.edn file that specifies a :git/url based dependency. I realize that this won't work with a pom.xml - does tools-deps find transitive dependencies using a deps.edn files in addition to pom.xml files?#2020-01-1719:37alexmillerYes, both, prefers deps.edn#2020-01-1719:38alexmillerOr rather will use exactly one but can be either#2020-01-1718:48borkdude@slipset there's also:
$ clojure -Sdescribe
{:version "1.10.1.496"
,,,}#2020-01-1719:26seancorfield@borkdude Are any of these differences between deps.clj and the regular clojure script of any concern?
PS C:\Users\sean\clojure\cool> deps -Sdescribe
{:deps-clj-version "0.0.6"
...
:config-files ["deps.edn" "C:\\Users\\sean\\.clojure\\deps.edn" "deps.edn"]
:config-user "C:\\Users\\sean\\.clojure\\deps.edn"
...
:install-dir nil
...
:force ""
:repro ""
...}
PS C:\Users\sean\clojure\cool> clojure -Sdescribe
{...
:config-files ["C:\Users\sean\scoop\modules\ClojureTools\deps.edn" "C:\Users\sean\.clojure\deps.edn" "deps.edn"]
:config-user "C:\Users\sean\.clojure\deps.edn"
...
:install-dir "C:\Users\sean\scoop\modules\ClojureTools"
...
:force False
:repro False
...}#2020-01-1719:27seancorfield:config-files seems odd with deps.edn in there twice (and :force/`:repro` seem to be "" but that's truthy?)#2020-01-1719:28seancorfield(I can understand :install-dir being nil since deps.clj is self-contained)#2020-01-1719:29borkdudeyes, that's the reason why install-dir is nil. the rest can be fixed if it's bothering anyone. also the escaping seems different?#2020-01-1719:30borkdudeI think False may be coming from Powershell? it doesn't seem like something cmd.exe would produce#2020-01-1719:32borkdude@seancorfield I may have put a str too many here: https://github.com/borkdude/deps.clj/blob/4c374238ceec510a097749df4b2f4a5a17a3e355/src/borkdude/deps.clj#L426#2020-01-1719:32borkdudeshould probably read boolean#2020-01-1719:33seancorfieldThe macOS/Linux version shows:
:force false
:repro false
#2020-01-1719:33seancorfieldFalse is just PS's version of false 🙂#2020-01-1719:33borkdudeI'll make an issue for this. It doesn't seem something that's very urgent, but I'll fix it#2020-01-1719:34seancorfieldI assumed the \\ was due to being produced by Clojure/Java, whereas \ is native PS (or cmd).#2020-01-1719:34borkdudeyeah, this is produced using pr instead of bash#2020-01-1719:35seancorfieldI guess :config-files is no longer used by any tooling so it doesn't matter -- it just looked odd with deps.edn in there twice.#2020-01-1719:35borkdudeyeah, might be indicative of a small bug somewhere.#2020-01-1719:36borkdudeI basically did the naive thing and ported bash straight into clojure without really think hard 🙂#2020-01-1719:38seancorfielddeps is a great option on Windows -- much appreciated!#2020-01-1719:39borkdudehttps://github.com/borkdude/deps.clj/issues/7
if you think this is fun, I also take PRs. there are build / dev instructions in the other open issue.#2020-01-1719:39alexmillerYeah Sean, get off the bench and help out for a change! ;)#2020-01-1719:40alexmillerI kid, I kid#2020-01-1719:40borkdude😄#2020-01-1719:40borkdudebtw bb.exe is now a thing on Windows so you can execute the deps.clj script as well (so you can tweak it in Notepad++)#2020-01-1719:41borkdude(I'll now stop the shameless promotion of my tools for the rest of the day)#2020-01-1719:43rschmuklerLooking through the source it doesn't look like transitive deps.edn files are supported in jars... Would a PR be welcome that adds this to the local resolver? similar to how find-pom works currently?#2020-01-1720:13alexmillerdeps.edn in jars is not a thing#2020-01-1720:14alexmillerand if I wanted to make it a thing, I'd try to design that first#2020-01-1720:26dominicmWhat would designing that look like, or do you mean that you would design it?#2020-01-1720:33alexmillerlike designing anything else - starting with goals, thinking about alternatives, examining their tradeoffs, making decisions, implementing#2020-01-1720:34alexmillerI don't have time to go down that path atm#2020-01-1805:15seancorfieldseancorfield/clj-new {:mvn/version "0.8.5"} -- generated lib and template projects now include :install and :deploy aliases, using @slipset’s excellent deps-deploy, so you can create a JAR clojure -A:jar, and either install locally clojure -A:install or deploy to Clojars clojure -A:deploy; also bumps depstar to 0.5.2 for the NPE bug fix in uberjars with no main class specified.#2020-01-1805:26seancorfieldMy dot-clojure/deps.edn file also has these aliases added (although they require a JAR filename as an argument, whereas they're baked into the generated lib and template projects since those already have a name).#2020-01-2002:52tapGot an error trying to start clj repl with rebl
Error building classpath. org.apache.maven.repository.internal.DefaultModelResolver.<init>(org.eclipse.aether.RepositorySystemSession, org.eclipse.aether.RequestTrace, java.lang.String, org.eclipse.aether.impl.ArtifactResolver, org.eclipse.aether.impl.VersionRangeResolver, org.eclipse.aether.impl.RemoteRepositoryManager, java.util.List)
java.lang.NoSuchMethodException: org.apache.maven.repository.internal.DefaultModelResolver.<init>(org.eclipse.aether.RepositorySystemSession, org.eclipse.aether.RequestTrace, java.lang.String, org.eclipse.aether.impl.ArtifactResolver, org.eclipse.aether.impl.VersionRangeResolver, org.eclipse.aether.impl.RemoteRepositoryManager, java.util.List)
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getConstructor(Class.java:1825)
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(pom.clj:49)
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invoke(pom.clj:39)
at clojure.tools.deps.alpha.extensions.pom$read_model.invokeStatic(pom.clj:60)
at clojure.tools.deps.alpha.extensions.pom$read_model.invoke(pom.clj:54)
at clojure.tools.deps.alpha.extensions.local$eval925$fn__927.invoke(local.clj:68)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at clojure.tools.deps.alpha$expand_deps$fn__1191.invoke(alpha.clj:191)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:190)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:168)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:236)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:216)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:56)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:48)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:82)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:73)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:102)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:96)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:147)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:119)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
deps.edn
{:deps {}
:aliases
{:rebl {:extra-deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/core.async {:mvn/version "0.4.490"}
com.cognitect/rebl {:local/root "/path/to/REBL.jar"}}
:main-opts ["-m" "cognitect.rebl"]}}}
The command I run
clj -R:rebl
Has anyone had the similar issue?#2020-01-2003:15seancorfieldWhat version does clojure -Sdescribe report?#2020-01-2003:17seancorfield@tap And you're aware that -R:rebl will add those dependencies but won't run REBL -- it will run the default behavior, which is to start a regular clojure.main REPL.#2020-01-2003:18tap{:version "1.10.1.496"
:config-files ["/usr/local/Cellar/clojure/1.10.1.496/deps.edn" "/Users/xxx/.clojure/deps.edn" "deps.edn" ]
:config-user "/Users/xxx/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/Cellar/clojure/1.10.1.496"
:config-dir "/Users/xxx/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
#2020-01-2003:18seancorfieldYour deps.edn file doesn't look valid -- you have five } on that line with com.cognitect/rebl#2020-01-2003:19tapSorry#2020-01-2003:19seancorfieldThat would put :main-opts outside the whole { } of the deps.#2020-01-2003:20tapYeah. Fixed up there. I removed :main-opts just to test it out. Miss that when I readded it to ask question here.#2020-01-2003:21tapYes. I'm aware that -R won't run REBL. Just want to make it clearer that it's not related to :main-opts.#2020-01-2003:22seancorfieldWhich version of REBL is that local JAR file?#2020-01-2003:23tapREBL-0.9.220.jar#2020-01-2003:25seancorfieldOk, I have that setup locally... just a sec...#2020-01-2003:25seancorfieldYeah, with -R:rebl I get that same error. I've just never run it that way -- that's odd.#2020-01-2003:26tapI got the same error with -A as well#2020-01-2003:29seancorfieldLet me switch computers and try something...#2020-01-2003:29tapI saw a quick fix related to pom for tools.deps.alpha recently. Not sure if it's related.
https://github.com/clojure/tools.deps.alpha/commit/fbe57d942af2016229e0aabba9121e62de848729
I'm also not sure if I'm using the latest tools.deps.alpha at the moment.#2020-01-2003:32seancorfieldCould be that or https://github.com/clojure/tools.deps.alpha/commit/70917267ca38fa971b1b0ebcc73084516f9f8af3#2020-01-2003:32seancorfieldI tried it with the add-lib branch SHA that I normally use and got the same exception.#2020-01-2003:33tapThanks for your help looking into this#2020-01-2003:35seancorfieldYeah, I can't start my regular dev environment with the latest clojure version with REBL (I've been on vacation for a several days so this is the first time I've tried since the last clojure/`tools.deps.alpha` update) -- /cc @alexmiller looks like the latest release breaks when used with REBL#2020-01-2003:43seancorfieldLooks like clojure isn't set up in a way that earlier releases can be installed with brew 😞#2020-01-2003:43seancorfieldbrew install fails#2020-01-2003:44sogaiu@seancorfield @tap it looks like i am able to reproduce the problem as well (this is a linux box)#2020-01-2003:45sogaiui have an install script for 1.10.1.492 i can try#2020-01-2003:46seancorfieldYeah, I think the latest changes in clojure/`tools.deps.alpha` are incompatible with the current release of REBL -- because it is AOT'd#2020-01-2003:46seancorfieldLooks like Cognitect will need to release a new version of REBL to fix this (which in turn won't be compatible with earlier clojure/`tools.deps.alpha` builds, I suspect).#2020-01-2003:47sogaiui just tried with 1.10.1.492 and it works with the REBL-0.9.220 jar#2020-01-2003:48sogaiuswitching to 1.10.1.496 gives the exception#2020-01-2003:49seancorfieldWeirdly, there's nothing I can see in the REBL JAR or its pom.xml that suggests it wouldn't be compatible... so I guess we have to wait for @alexmiller and/or some other Cognitect folks to chime in tomorrow. Very frustrating we can't downgrade clojure via Homebrew tho'...#2020-01-2003:52sogaiuindeed#2020-01-2003:54sogaiui guess the bit about "installing from a specific commit" here: http://effectif.com/mac-os-x/installing-specific-version-of-homebrew-formula doesn't apply?#2020-01-2003:56sogaiumay be the info is dated...#2020-01-2003:59seancorfieldWorkaround:
$ curl -O
$ curl -O
$ tar xvfz clojure-tools-1.10.1.492.tar.gz
and then edit your current clojure shell script to point to the clojure-tools/clojure-tools-1.10.1.492.jar from that expansion instead of the installed 496 version.#2020-01-2004:00seancorfield(at least I'll be back up and running at work tomorrow morning now!)#2020-01-2004:01alexmillerthe exception above from @tap is a regression that is fixed in master and I'll release it tomorrow#2020-01-2004:02alexmilleryou can actually get the older brew formulas if you want to download#2020-01-2004:02alexmillerat https://download.clojure.org/install/clojure-1.10.1.492.rb#2020-01-2004:04alexmillerrename that to clojure.rb, then brew reinstall --force clojure.rb#2020-01-2004:04alexmillerbrew can only reinstall older versions that you have previously downloaded and installed that you still have in the brew cache#2020-01-2004:05alexmillerthere is no repository of versions like there is with Maven or something, just "the current version"#2020-01-2004:11seancorfieldIs there an option on upgrade or in a specific formula to not delete the old version?#2020-01-2004:11seancorfield(and good to know on the force install from a downloaded older version)#2020-01-2004:16alexmillerby default, I think it leaves it#2020-01-2004:18alexmillerls ~/Library/_Caches_/Homebrew#2020-01-2004:19alexmillerand then Formula/clojure under that#2020-01-2004:19tapIs it appropriate to write about this how to downgrade brew clj package on https://clojure.org under Guides section? I'll do that if y'all think so.#2020-01-2004:20alexmillernah, I'd prefer to bury it in reference and link#2020-01-2004:21tapok#2020-01-2004:43seancorfieldUnder ~/Library/Caches/Homebrew is just the current version of each installed package.#2020-01-2019:15alexmillerLatest (1.10.1.502) with regression fixes is now available in brew etc#2020-01-2019:15alexmillerFixed for pom deps and pom gen#2020-01-2019:18seancorfieldWorks with REBL now -- thank you for the swift update!#2020-01-2019:21alexmillerI also added some tests to catch it earlier next time!#2020-01-2020:48sogaiuseems to work on a linux here 👍#2020-01-2021:25alexmilleroh also, this release has the version as the first line of clj -h#2020-01-2021:43andy.fingerhutYou have been worn down explaining the -Sdescribe and -Sverbose options 🙂 Soon you will succumb to the pull to the dark side of adding a --version option.#2020-01-2021:58alexmillereven that won't be the end - there will be confusion between the clojure tools version, the clojure tools default clojure version, and the version of clojure specified by the current project :)#2020-01-2022:03andy.fingerhutLong ago I gave myself slightly more sanity by including a line like the following at the beginning of Eastwood's output, by default: == Eastwood 0.3.6-SNAPSHOT Clojure 1.10.0 JVM 1.8.0_192 ==#2020-01-2022:30seancorfieldHeh, that's why all our services report their git tag/SHA, along with the version of Clojure and the JVM at startup! 🙂#2020-01-2021:29littleliseems to work on Windows too#2020-01-2021:58borkdudehttps://github.com/borkdude/deps.clj/releases/tag/v0.0.7#2020-01-2115:54aviHas anyone here ever managed to get Snyk to scan a Clojure project using tools.deps for vulnerabilities? I’m giving it a try and Snyk is helping out but I’m running into Maven problems.#2020-01-2115:54avi[ERROR] Failed to execute goal on project project: Could not resolve dependencies for project project:project:jar:0.1.0: The following artifacts could not be resolved: com.gfredericks:test.chuck:jar:0.2.10, clj-commons:clj-yaml:jar:0.7.0, hawk:hawk:jar:0.2.11: Could not find artifact com.gfredericks:test.chuck:jar:0.2.10 in central () -> [Help 1]
#2020-01-2115:55aviI thought maybe I was running into this: https://github.com/snyk/snyk/issues/207#2020-01-2115:55aviBut I checked and the pom.xml generated by clojure -Spom already includes both Central and Clojars.#2020-01-2115:55aviSo I’m stuck…#2020-01-2115:59alexmillercould you post the pom?/#2020-01-2115:59avisure#2020-01-2116:00aviJust re-running the CI job now to regenerate it.#2020-01-2116:03avi#2020-01-2116:04aviDoh! picard-facepalm repositories is empty#2020-01-2116:04aviNever mind!#2020-01-2116:05aviOr… is there a way to get clojure to populate repositories?#2020-01-2116:05aviI’m using 1.10.1.483 but could switch to something newer#2020-01-2116:07alexmillerI'm trying to remember if it's supposed to do that now or not#2020-01-2116:07aviI’m going to try it with 1.10.1.502 and report back#2020-01-2116:07alexmillerlooks like it is, so might be broken#2020-01-2116:10avi😅#2020-01-2116:11alexmillerI guess it will only right now add repos explicitly listed in the deps.edn#2020-01-2116:11aviI’ve got both repos explicitly in deps.edn#2020-01-2116:12aviI’m testing locally now with 1.10.1.502#2020-01-2116:12avirepositories is empty 😅#2020-01-2116:12avimy deps.edn has this at the root:
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}#2020-01-2116:15alexmillerI'm looking at it#2020-01-2116:16avithanks2#2020-01-2116:20alexmillerah, it's subtle but I see the problem#2020-01-2116:20avibananadance#2020-01-2116:25aviany possible workaround? Or maybe I should brush up on my sed skills and try to just patch the pom for now?#2020-01-2116:25alexmillerfor the moment you'll need to manually modify your pom.xml after gen#2020-01-2116:26aviah ok#2020-01-2116:26aviwill do — thanks!#2020-01-2116:26aviWould you like a bug report somewhere?#2020-01-2116:26alexmillernah, I've got it#2020-01-2116:26avi💪 thanks3#2020-01-2117:27alexmillerfixed for next release, but not sure when that will be (and added a test for future)#2020-01-2117:28aviFantastic — thanks so much!#2020-01-2118:59danielcomptonIs there a way to get tools.deps to print out all of the :paths and :extra-paths that a given set of aliases resolve to?#2020-01-2119:00danielcompton-Sdescribe prints out some information, but nothing about the composed config#2020-01-2119:01alexmillernot currently, although I've been making my way towards making something like that possible#2020-01-2119:01alexmillerI've wanted something like mvn help:effective-pom so that's been a back burner project of mine#2020-01-2119:01alexmillerdo you care about :paths vs :extra-paths?#2020-01-2119:02alexmilleror just "all local paths"#2020-01-2121:45danielcomptonAll local paths for my usage (send them to clj-kondo)#2020-01-2121:48alexmillerok, that's good#2020-01-2119:37weavejesterDoes anyone know if there’s an uberjar library for tools.deps that supports merging of files like data_readers.clj?#2020-01-2119:38ghadidepstar does it#2020-01-2119:38ghadimaybe?#2020-01-2119:38ghadiIt can be taught to, easily#2020-01-2119:38weavejesterI looked into depstar, but it looks like it has hardcoded that part. I could submit a PR, but I was hoping there was something around already.#2020-01-2119:39ghadihttps://github.com/seancorfield/depstar/blob/master/src/hf/depstar/uberjar.clj#L57-L61#2020-01-2119:39ghadihave you tested it @weavejester ?#2020-01-2119:40weavejesterYeah, that’s what I mean by it being hard-coded, @ghadi.#2020-01-2119:40weavejesterNotice data_readers.clj is explicitly checked for.#2020-01-2119:41weavejesterAnd clash-strategy doesn’t look like it can be overridden.#2020-01-2119:41ghadioh you meant other files, not that one in particular#2020-01-2119:41weavejesterYeah#2020-01-2119:42ghadinot extensible as written#2020-01-2119:55dominicm@weavejester pack avoids merging altogether.#2020-01-2119:55dominicmCapsule is the one that works with most of your libraries :)#2020-01-2120:14seancorfield@weavejester Happy to entertain PRs on depstar to extend the clash handling logic. I inherited that as-is and haven't enhanced that part yet...#2020-01-2123:12weavejesterAre there any disadvantages to Capsule jars? I guess I’m inherently wary of anything that builds on known deployment mechanisms.#2020-01-2207:28dominicmThey use a little bit more memory because they start trampoline jvm, much like lein does#2020-01-2207:28dominicmOnejar is a little more special, that uses a class loader to do its work#2020-01-2207:28dominicmYou might also want to consider the docker mode, which will directly create a docker image.#2020-01-2306:26tianshu@U09LZR36F can I package a clojure project into a docker image?#2020-01-2307:43dominicmYeah#2020-01-2308:34tianshu@U09LZR36F can you share your workflow?#2020-01-2308:38dominicm@U0NBGRGD6 My understanding is that you provide a docker registry and it will upload it there. People are using it with Google.#2020-01-2311:55rickmoynihan@weavejester: Yes there are definite disadvantages to capsule jars…
Also they extra the jars contents into a cache to build the classpath.
You also get a jvm dance at startup; and you then need to care about proxying various JVM options to the real app vm; i.e. if you want to set the heap etc.
I’ve been meaning to look at onejar as an alternative.
FWIW in spite of the occaisional frustration we currently avoid the issue you’re on about with capsule jars. I think there’s an argument that merging the files has problems too; it would be better to resolve all of them at runtime; i.e. uberjars are really just a hack.
I just wish the JVM platform itself supported this.#2020-01-2311:57rickmoynihanIt might just be better to zip up the classpath and write it in a shell script that starts the jvm tbh.#2020-01-2311:58rickmoynihanIIRC I wrote this stuff up on a duct issue somewhere… 👀#2020-01-2311:58rickmoynihanhttps://github.com/duct-framework/duct/issues/92#issue-449273982#2020-01-2312:02rickmoynihanAlso lein-tools-deps and using lein for just the packaging and deployment may also be a solution for you. (Full disclosure: I wrote it)#2020-01-2314:00dominicmIf I had control over how people deployed, I'd tell them to use the skinny mode, aka, dump my project into a folder#2020-01-2314:01dominicmThere's an idea on what you just said though#2020-01-2314:01dominicmZip will ignore the prefix leading up to finding the header. So you can have zips which are scripts.#2020-01-2314:02dominicmUnzip itself to a cache directory, and pass all arguments to the jvm#2020-01-2204:54Nate Suttonwhere are git deps cloned to?#2020-01-2205:00Nate Suttonfound it, ~/.gitlibs/libs#2020-01-2205:01Nate Suttonwell, and ~/.gitlibs/_repos#2020-01-2213:00alexmillerThe object dirs and working trees are split there#2020-01-2310:24sveriHi, is there a way to define an alias that extends an existing alias?#2020-01-2310:30vlaaadno alias composition, as far as I know#2020-01-2310:34sveriOk, thank you 🙂#2020-01-2320:49samAre private git repos supported in deps.edn? The documentation says public and private are supported, but I'm not sure how to provide credentials for private and I can't find any instructions.#2020-01-2320:53samNvm, I think I found documentation here: https://clojure.org/reference/deps_and_cli#_procurers.#2020-01-2320:55alexmilleryes and yes#2020-01-2320:55alexmillerit can be a bit fiddly though, so when it doesn't work, come back here :)#2020-01-2320:58samThanks!#2020-01-2321:09samI got the private repo part to work, but got this error:
Error building classpath. Manifest type not detected when finding deps for provider-data-pipeline/provider-data-pipeline
which is apparently due to the fact that that the target repo is using leiningen for dependency management instead of deps.edn. Is that right?#2020-01-2321:14seancorfield@samwagg0583 Correct. t.d.a can only process deps.edn and pom.xml based git dependencies.#2020-01-2321:16borkdudeif it's a dependency that only has sources and no other deps in its project.clj you can declare a manifest type:
Optional key :deps/manifest
Specifies the project manifest type
Default is to auto-detect the project type (currently either :deps or :pom)
This works for e.g. libs like medley#2020-01-2402:25aviI like this tip! Thanks!#2020-01-2321:19seancorfieldYes, true, you can "force" :deps and it treats it as if deps.edn exists with {} as its contents.#2020-01-2321:30samThanks everyone.#2020-01-2321:54samWhat about using lein pom to generate a pom and committing that to the target git repo? I do have control over the target, but I might not be able to convert it to deps.edn, at least in the short term. Curious if you guys have heard of anyone using that pattern and how it's worked out.#2020-01-2321:56samI mean, I'd rather not have 2 different listings of deps like that. But it seems like an option.#2020-01-2322:09seancorfield@samwagg0583 If the repo has dependencies, then adding a pom.xml file seems like a good compromise.#2020-01-2322:09seancorfieldJust bear in mind that if someone updates project.clj and forgets to update pom.xml you'll have inconsistent dependencies downstream.#2020-01-2322:10seancorfieldAnother alternative would be to put the basic dependencies from project.clj into deps.edn in that repo and use a plugin to pull them in from that file...#2020-01-2322:10seancorfieldhttps://github.com/RickMoynihan/lein-tools-deps lets you do that#2020-01-2322:11seancorfieldSo then deps.edn would be the "system of record" for the primary dependencies and both the Lein project and downstream consumers via git will see the same file, same deps.#2020-01-2413:49rickmoynihan@samwagg0583 author of lein-tools-deps here. It does let you do what @seancorfield says; though I wouldn’t recommend ltd for projects that have complex transitive downstream dependencies.
The reasoning is covered in the README here: https://github.com/RickMoynihan/lein-tools-deps#do-you-really-want-to-use-this
However there is an open issue to fix this, and a known solution for fixing it: https://github.com/RickMoynihan/lein-tools-deps/issues/91
(As an aside I’m unlikely to find the time to fix this myself anytime soon; but am open to PRs). I think this is the most important issue to fix in that project.#2020-01-2822:09sam@U06HHF230 Thanks for the info!#2020-01-2822:12samFor the moment, I went the route of publishing a pom.xml in the target repo. But I may explore your lib more in the future.#2020-01-2414:22holyjakHello! Does anyone have an example how to create a simple clojure + java based project with deps.edn? (Clojure will use some Java classes). Quick googling revealed nothing relevant...#2020-01-2414:32alexmillerdeps does not explicitly support Java compilation#2020-01-2414:33alexmillerso you either need something that can combine deps with javac, or split the project into java part and clojure part#2020-01-2415:04sogaiu@holyjak if you haven't checked out @mike1452's recent 2 announcements for the clj-new templates slib and sapp, may be it's worth a look?#2020-01-2416:00mike_ananev@holyjak try this
https://github.com/mikeananev/sapp#2020-01-2416:04holyjakThank you!#2020-01-2416:02alexmillermoving here from announcements sub-thread. @seancorfield I'm seeing an error trying to use clj-new with that sapp template:#2020-01-2416:02alexmillerclj -A:new qwerty/asd
Failed with: org.apache.maven.repository.internal.DefaultModelResolver.<init>(org.eclipse.aether.RepositorySystemSession,org.eclipse.aether.RequestTrace,java.lang.String,org.eclipse.aether.impl.ArtifactResolver,org.eclipse.aether.impl.VersionRangeResolver,org.eclipse.aether.impl.RemoteRepositoryManager,java.util.List)
Execution error (ExceptionInfo) at clj-new.helpers/resolve-remote-template (helpers.clj:149).
Could not load artifact for template: slib
Tried coordinates:
[slib/boot-template "RELEASE"]
[slib/lein-template "RELEASE"]
Full report at:
/var/folders/2k/2l7ch1xn7lb5gj86tgp1ygww0000gn/T/clojure-13105343161273067757.edn
#2020-01-2416:03alexmillerThe DefaultModelResolver thing is either the problem I just fixed in latest tools.deps.alpha or a version conflict due to pomegranate (both kind of looked similar errors). I notice that clj-new is using a pretty old version of tools.deps?#2020-01-2416:06mike_ananevhmm, interesting. i just checked out template on clear centos 7 with latest clj-new 0.8.5 and tools deps. everything works fine. can't reproduce DefaultModelResolver error.#2020-01-2416:06alexmillerMy clj-new alias is
:new {:extra-deps {seancorfield/clj-new {:mvn/version "0.8.5"}}
:main-opts ["-m" "clj-new.create"]}#2020-01-2416:07mike_ananevso, my alias looks the same:
:new {:extra-deps {seancorfield/clj-new {:mvn/version "0.8.5"}}
:main-opts ["-m" "clj-new.create"]}#2020-01-2416:08alexmillerif it's the conflict with transitive pomegranate maven deps, that may be the kind of thing that is arbitrary and resolves differently on different machines. can you dump me clj -Stree -A:new ?#2020-01-2416:08alexmillerso I can compare#2020-01-2416:09alexmillerand also, what version of clj are you on (`clj -Sdescribe`)#2020-01-2416:10mike_ananev{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "f79a9e9c92b79a9587c7c5"}, :content ("[email protected]")}#2020-01-2416:16alexmillerthat was same for me#2020-01-2416:16alexmillerbumping to latest tools.deps with clj-new worked#2020-01-2416:16alexmiller:new {:deps {seancorfield/clj-new {:mvn/version "0.8.5"}
org.clojure/tools.deps.alpha {:mvn/version "0.8.640"}}
:main-opts ["-m" "clj-new.create"]}`#2020-01-2416:17alexmillerprobably because I'm not calling the DefaultModelResolver at all in latest#2020-01-2416:17alexmillerI guess @seancorfield that I would suggest maybe bumping tda in clj-new#2020-01-2416:20mike_ananevI will add org.clojure/tools.deps.alpha {:mvn/version "0.8.640"} as recommendation to README in case of errors#2020-01-2416:22alexmillerwell, I'd hold off on that, I hate to cargo cult workarounds like that#2020-01-2416:22alexmillerI could file an issue?#2020-01-2416:23alexmillerI work with a hacked up clj environment a lot of the time and this may truly be just something on my machine (although I think I have undone all of that)#2020-01-2416:27seancorfield@alexmiller Can you open an issue on clj-new to bump the deps? That way I won't forget when I get to work, and I'll cut 0.8.6 today.#2020-01-2416:27alexmillersure#2020-01-2416:32alexmillerdone#2020-01-2417:05seancorfield@mike1452 seancorfield/clj-new {:mvn/version "0.8.6"} is up on Clojars -- can you try that out (without adding t.d.a.) and see if it resolves your issues?#2020-01-2417:06seancorfieldAlso @alexmiller 🙂#2020-01-2417:07seancorfieldI also bumped Pomegranate to the latest version, which is on clj-commons now, which may (or may not) cause new problems since the group ID has changed and code will possibly get two versions of it...#2020-01-2417:09seancorfield(the Jackson dependency hell conflicts with various templates are why 2.7.5 of that is also pinned in clj-new since Luminus' template was breaking before -- due to it relying on Leiningen's version resolution avoiding a conflict there)#2020-01-2417:24alexmillerbtw, as of the new s3 transporter in tda, it no longer has a jackson dep#2020-01-2417:33seancorfieldOK, I may remove that and see if Luminus or other project templates still break.#2020-01-2417:34alexmilleryou may still be pulling it in from multiple places, but tda is not one of them :)#2020-01-2417:36seancorfieldLuminus templates seem to generate OK without clj-new's deps on Jackson so in 0.8.7 I'll take them out.#2020-01-2417:36alexmilleralso, you probably want the clj-new alias to be :deps, not :extra-deps so that you override, instead of add to, the deps of the current project deps.edn if there is one#2020-01-2417:36seancorfieldGood point. That wasn't available before.#2020-01-2417:36alexmillerusually it probably doesn't matter as you're making a new project#2020-01-2417:36alexmiller(but note that this requires a sufficiently new clj version, and you'll get an error if using older)#2020-01-2417:37alexmillerI'd recommend at least clj 1.10.1.483 (Nov 2019 vintage)#2020-01-2417:38alexmillerso maybe that's something to hold off on#2020-01-2417:41seancorfieldYeah... I'll give it a month or two and a few more releases of clj before I make that change. I just saw someone still using clj-new 0.5.5 😐#2020-01-2417:16mike_ananev@seancorfield I'll check. just a moment.#2020-01-2417:22alexmiller@seancorfield worked#2020-01-2417:23defaI have a tools-deps based project with a Clojure backend (lacinia-pedestal) und a re-frame frontend (ClojureScript). So it is a mixed Clojure/ClojureScript project. I’m using :aliases to run the server and the frontend (via shadow-cljs). Am I supposed to define the deps as :extra-deps in the related aliases? I just debugged an issue where jetty stopped working with very strange exceptions and that was related to having pedestal-lacinia and re-graph in :deps which caused incompatible versions of jetty libraries loaded. Moving the re-graph dep into the frontend alias solved the problem.#2020-01-2417:25alexmillerif you have deps that are only needed when the alias is enabled, then they should be :extra-deps in that alias#2020-01-2417:26alexmillersorry if I misinterpreted the question#2020-01-2417:28defaI guess that’s the answer, since it is working. Is it in general a good/bad idea to have client and server in one deps.edn?#2020-01-2417:30alexmillerI don't think it's either good or bad#2020-01-2418:27dominicmTrue neutral?#2020-01-2418:28alexmiller💯#2020-01-2418:28alexmilleror maybe lawful neutral#2020-01-2418:51defaAt least it is very convenient when working with cursive inde IntelliJ IDEA. Guess I really have to give emacs a try one day…#2020-01-2421:12dominicmFwiw, I find it equally useful with vim, and my colleagues do with emacs.#2020-01-2421:13dominicmI try to defer splitting projects as long as possible. Development is faster when you don't have to fabricate arbitrary boundaries between things where you haven't found the boundary yet#2020-01-2517:35sveriWhile I agree with this sentiment in general, dealing with dependencies will cause troubles sooner or later (not only in clojure / java, but development in general).
If I have learned something over the decades it's these rules.
1. Uses as few dependencies as possible.
2. Separate these as much as possible.
3. Use the same version of a dependency everywhere, if you can.
4. Avoid gradle and eclipse, as eclipse still cannot separate classpathes like gradle does.#2020-01-2518:28dominicmTools deps has made my life significantly better in this space, so has the clojure culture of not breaking the api#2020-01-2417:40mike_ananev@seancorfield @alexmiller I have updated templates on GitHub with clj-new 0.8.6+#2020-01-2722:56salami hope this is the right channel to ask questions about deps.edn and the clojure cli tools.
i was wondering whether deps.edn can be spec'ed so that if one makes a mistake (e.g., putting a string literal where a symbol is expected), the cli tools can give some kind of feedback on the said mistake as spec errors. has the core team considered this?#2020-01-2723:30andy.fingerhutThis is a reasonable place to ask such questions about those tools. I'm not part of the core team, and do not know their views on stricter checking of deps.edn file contents.#2020-01-2723:49salamthanks for the response, @U0CMVHBL2 🙂#2020-01-2723:39alexmillerThere are specs in the tools.deps repo#2020-01-2723:40alexmillerIt’s not currently checked as we were trying to support older Clojure versions although we could probably bump that req now#2020-01-2800:00salamaha, indeed:
https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/specs.clj
yeah, being able to leverage this in clojure 1.10.x (or later versions) would be great.
i'll settle for the comment form at the bottom of that spec file for now. 🙂 thanks!#2020-01-2800:01alexmillerYeah, I used that to validate all the deps.edn I had lying around :)#2020-01-2822:55alexmillerso I kind of got distracted into implementing parallel downloads for tools.deps today#2020-01-2822:56alexmillermight need some hearty volunteers to try it before I release it though#2020-01-2822:57alexmillerit was one of those things where I got it working in about 20 minutes, then spent 3 hrs properly rewriting it to account for error handling and all the other real work#2020-01-2822:57alexmillerbut it's definitely faster when you're downloading a bunch of junk#2020-01-2823:00alexmillerit all of a sudden got a lot slower a while ago and I thought I had broken something until I realized my son was downloading a giant game 😄#2020-01-2823:32seancorfieldHow should we test it for you?#2020-01-2823:44alexmillerwell, I'm still hacking on it a bit, but I'll try to make something available#2020-01-2904:10dharriganHappy to volunteer too!#2020-01-2904:48alexmillerthe interesting bit is that parallelizing the jar downloads only got me so far because with Clojure artifacts, the pom is of significant size compared to the jar files and we end up downloading a lot more metadata during version selection#2020-01-2904:50alexmillerso, working through parallelizing the metadata download too, which is a lot harder to work into the algorithm#2020-01-2904:50alexmillerbut, still seems doable#2020-01-2907:50dominicmVery cool. Does maven do parallel at all?#2020-01-2907:50alexmilleryeah#2020-01-2907:51alexmillerI have parallel metadata during expansion working now#2020-01-2907:51alexmillerthe benefits vary a lot depending on how much metadata/how much data/how much parallelism/how bushy the dep tree is, etc#2020-01-2907:52alexmillerfor something like the luminus template, for lein to download the full dep tree is ~43 sec#2020-01-2907:52alexmillerclj before was 73 sec and is now 10 sec#2020-01-2907:52alexmillerthat's kind of an extreme (lots of jars, huge fanout, lots of java, so big jars) etc#2020-01-2907:53alexmillerbut 20-60% improvement is a common range#2020-01-2907:54alexmillermost of the time of course you're not really seeing those full numbers because you already have some or most of the libs downloaded in the cache#2020-01-2909:15p-himikIt seems that "RELEASE" and "LATEST" are not available in Maven 3, but tools.deps seem to support them.
However, I cannot find any mention of these aliases in the documentation.
What's the deal with them? Can people rely on them? Can developers safely recommend using them for e.g. such things as tools in the user's deps.edn?
Also, I know that "RELEASE" is susceptible to cache issues - if no new dependency is needed to be retrieved during dependency resolution, the latest release available locally will be used (i.e., no check for a new release will be made) or something like that.
Is "LATEST" also susceptible to such issues? Or does it reliably make a check for a new version each time the dependency resolution is done?#2020-01-2913:26alexmillerThey are not supported and break the cache checking#2020-01-2913:27alexmillerThat said, they do actually work and can be useful for one off calls in something like -Sdeps#2020-01-2913:28alexmillerI recommend using real versions instead#2020-01-2913:30p-himik@thheller ^ FYI
Yeah, I use "RELEASE" only for clj-kondo, tools.deps.graph, etc. Something that runs only to facilitate the development.#2020-01-2915:01alexmillernote that if you use RELEASE for tools, you generally won't get new versions of the tools after you first run them, unless you -Sforce (because the hashed key is just seeing "RELEASE" every time)#2020-01-2917:18seancorfieldGood to know. Explains some of the slightly odd behavior I thought I was seeing from time to time since my dot-clojure deps.edn uses RELEASE a lot.#2020-01-2917:19seancorfieldclj-new has moved away from that and tracks explicit versions now.#2020-01-2917:20seancorfieldIt would be nice if there was some way in deps.edn to flag a dependency that should be rechecked periodically rather than cached indefinitely.#2020-01-2917:24alexmillerThe issue is that the cache check is done in bash to avoid jvm load so the hash is purely text based#2020-01-2917:25alexmillerSo there is no easy way to do what you’re suggesting without a lot of trade offs#2020-01-2917:27seancorfieldYeah, I think it would need an additional file in .cpcache for "dependencies that should be rechecked" which is probably more complexity than anyone would want for the occasional use of this feature.#2020-01-2917:31seancorfieldIs it worth me spending time thinking about this problem or would such a proposal not be considered anyway?#2020-01-2918:43alexmillerWay down my list#2020-01-2918:46seancorfieldI'll take that as a "no" then 🙂#2020-01-2915:02alexmillerprobably you're changing your local deps.edn or clj install frequently enough that you are periodically re-caching and don't notice this#2020-01-2915:21borkdudeit would be cool if tools.deps could resolve RELEASE in addition to tags maybe?#2020-01-2915:22borkdudeas in replace RELEASE with the most recent version in your deps.edn file#2020-01-2915:35alexmillerI'm not planning on adding any more mucking with deps.edn#2020-01-2915:36alexmillerbut does not need to be in tools.deps - this itself could be a tool#2020-01-2915:36alexmillerI'm not sure that's actually the best solution though#2020-01-2915:38dharriganhttps://github.com/Olical/depot does something similar, although I do not know if it processes "RELEASE" versions, but it updates the deps.edn in place definitely for deps that have versions.#2020-01-2915:44dharriganMaybe it does Besides SNAPSHOT versions --resolve-virtual will also handle the special version strings "RELEASE" and "LATEST"#2020-01-2915:51borkdudeThis script uses RELEASE to print the most recent versions of libraries that are newer than you current ones:
https://github.com/borkdude/babashka/#list-outdated-maven-dependencies
A bit of a hacky light version of depot (thanks seancorfield)#2020-01-2916:23robert-stuttafordcan i remove a dependency in an override?#2020-01-2916:24alexmilleriirc, no, but this has been on my radar for a while#2020-01-2916:25robert-stuttafordI have datomic-pro overriding my tools.reader dep#2020-01-2916:25robert-stuttafordlooks like i'll have to use a cljs alias#2020-01-2916:26thhellernot exactly overriding. it contains a AOT compiled version of tools.reader which clojure will load instead of the actual tools.reader version on the classpath#2020-01-2916:26alexmillerthat's been fixed in datomic-pro, but not sure if that's out yet#2020-01-2916:29alexmilleryeah that fix is in 0.9.6024#2020-01-2916:30alexmillerhttps://forum.datomic.com/t/datomic-0-9-6024-now-available/1310#2020-01-2916:41robert-stuttafordthanks, @alexmiller!#2020-01-2916:24rickmoynihan@alexmiller: I vaguely remember you once mentioning a tools.deps top level keyword for tooling to use that wouldn’t pull in the main :deps? I can’t see any mention of it, so I’m guessing it’s not implemented yet?#2020-01-2916:25borkdudeyou actually can remove a dependency with the classpath overrides#2020-01-2916:25alexmillerif you are in an alias, you can use :deps to replace instead of :extra-deps to add#2020-01-2916:25rickmoynihanahh that was it, thanks.#2020-01-2916:25borkdude@robert-stuttaford https://github.com/borkdude/babashka/issues/152#2020-01-2916:26borkdude^ setting the classpath override to nil will remove it from the classpath#2020-01-2916:27robert-stuttafordoh man thank you#2020-01-2916:27borkdudeonly works in an alias I think#2020-01-2916:28thhelleror you could have a server/clj alias so cljs alias doesn't inherit all the other deps as well#2020-01-2916:31robert-stuttaford@borkdude's suggestion worked! :cljs {:classpath-overrides {com.datomic/datomic-pro nil}}#2020-01-3010:17onetomi have some issues suddenly downloading the expectations/clojure-test dependency:
% clj -Aexpect
Downloading: expectation/clojure-test/1.2.1/clojure-test-1.2.1.pom from
Downloading: expectation/clojure-test/1.2.1/clojure-test-1.2.1.jar from
Error building classpath. Could not find artifact expectation:clojure-test:jar:1.2.1 in central ()
my project's deps.edn explicitly specify these repos:
:mvn/repos
{"clojars" {:url ""}
"central" {:url
""
#_""}
"" {:url ""}
"datomic-cloud" {:url ""}}
im not sure how to debug the situation#2020-01-3012:54practicalli;; - see https://github.com/clojure-expectations/expectations
;; - and https://github.com/clojure-expectations/clojure-test
;; - run your expectations: clj -A:test:expect:test-runner (using cognitect test runner)
:expect
{:extra-deps {expectations {:mvn/version "RELEASE"}
expectations/clojure-test {:mvn/version "RELEASE"}}} #2020-01-3012:55practicalliIts from a collection of common deps I keep here https://github.com/practicalli/deps-edn-examples/blob/master/deps.edn#2020-01-3013:00practicalliMy blog has more up-to-date info about deps.edn and the Clojure CLI, I havent moved that to the book yet
https://practicalli.github.io/blog/archives/
See July and August articles#2020-01-3013:00practicalliThe #tools-deps channel is also useful for deps.edn issues#2020-01-3013:01practicalliYou can also send me a direct message if you need help and cant find where to get help#2020-01-3010:19borkdudeMaybe you have a typo, expectation vs expectations?#2020-01-3010:19onetommy ~/.clojure/deps.edn also contains one repo:
:mvn/repos {
"datomic-cloud" {:url ""}
}
how are this per-user and per-project :mvn/repos are getting merged?#2020-01-3013:52alexmillerJust that - they are merged, project merged over user merged over built in#2020-01-3010:20onetomindeed, thanks @borkdude!#2020-01-3010:22onetomi tried to copy out the deps specification from a remote, terminal based emacs, running in an iterm2, but it has mouse support, so i could only yank it into the remote emacs' kill ring...
that's why i retyped it locally... incorrectly...#2020-01-3010:25onetomim battling with emacs for months now, slowly learning more and more about it but i still end up falling back to cursive to get things done...
im really wondering if there is some easy way to get into clojure development with emacs or everyone goes through these serious birthing pains?
i found https://practicalli.github.io/clojure/ recently which seems promising#2020-01-3010:28borkdude@onetom This is a bit off topic for this channel, but feel free to continue about learning clojure in #beginners#2020-01-3010:28borkdude(or #emacs, #off-topic , etc)#2020-01-3014:29dharriganthat graph view for deps is aaaammmmaaaaazing 🙂#2020-01-3014:44maleghastgraph view..? I think that I may have missed something, but scrolling back I can't find it... 😞#2020-01-3015:22dominicmTools.deps.graph#2020-01-3015:22dominicmOn github#2020-01-3015:23shaun-mahoodhttps://github.com/clojure/tools.deps.graph#2020-01-3014:29dharriganThanks Alex!#2020-01-3014:32alexmillertry it with --size too#2020-01-3015:30kszaboSadly --size doesn’t work with gitlibs based monorepo dep that uses :deps/root specified deps.edn which references a sibling monorepo folder dep via :local/root "../aws" . Quite the contrived example. Here’s the obfuscated stacktrace:
/Users/kszabo/.gitlibs/libs/monorepo/aws/8badc505955040c6a8780bfab4e0484ca4c69a45/aws is not a relative path
java.lang.IllegalArgumentException: /Users/kszabo/.gitlibs/libs/monorepo/aws/8badc505955040c6a8780bfab4e0484ca4c69a45/aws is not a relative path
at $as_relative_path.invokeStatic(io.clj:414)
at $file.invokeStatic(io.clj:426)
at $file.invoke(io.clj:418)
at clojure.tools.deps.alpha.extensions.git$eval1027$fn__1029.invoke(git.clj:43)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.graph$get_size.invokeStatic(graph.clj:95)
at clojure.tools.deps.graph$get_size.invoke(graph.clj:93)
at clojure.tools.deps.graph$make_dep_node.invokeStatic(graph.clj:107)
at clojure.tools.deps.graph$make_dep_node.invoke(graph.clj:99)
at clojure.tools.deps.graph$make_graph$fn__1522.invoke(graph.clj:125)
#2020-01-3015:44alexmilleryeah, that came up recently, will fix on next rev#2020-01-3021:08alexmilleris now fixed in tools.deps.graph 0.2.30#2020-01-3016:03dominicmIf I have a main that takes arguments that look like clojure's (e.g. pack) and I run like so:
clojure -A:pack -- -A:for-pack-pls
Then both the -- and the -A:for-pack-pls are passed to pack, but I'd only expect the -A:for-pack-pls to be passed.#2020-01-3016:06dominicmOh, looks like it's because -- is an unknown flag to the cli, so it just passes it along... It would be handy if it actually followed the -- convention itself.#2020-01-3016:07dominicmOpen to a patch?#2020-01-3016:10borkdudeI was also considering this for the irrelevant unclojure.#2020-01-3016:11borkdudeCould also be built into https://github.com/borkdude/deps.clj/#2020-01-3016:12dominicmWhat's unclojure? 😄#2020-01-3016:13borkdudeyou know, the off-topic one 🙂#2020-01-3016:14dominicmHaha, yes.#2020-01-3018:59borkdudehttps://github.com/borkdude/babashka/issues/250#2020-01-3016:19alexmilleris that a question for me re tools.deps or for pack?#2020-01-3016:19borkdudetools.deps#2020-01-3016:19alexmillersorry, I don't understand what's being requesteed#2020-01-3016:19alexmillerwhat is the -- convention?#2020-01-3016:20borkdudethe arguments after -- should be passed as *command-line-args*#2020-01-3016:20borkdudes/should/could#2020-01-3016:20alexmillerso is this really a clojure.main request?#2020-01-3016:20dominicm@alexmiller for you.#2020-01-3016:20dominicmNo, this is a tools.deps request. The patch would be to the clojure script#2020-01-3016:20borkdudethe only tool where I've seen this is the docker cli#2020-01-3016:20borkdudebut I bet there are many others#2020-01-3016:21dominicmgit uses it too, in case you have files starting with -#2020-01-3016:21dominicmdiff --git a/src/main/resources/clojure b/src/main/resources/clojure
index 851d1a6..c6170a3 100755
--- a/src/main/resources/clojure
+++ b/src/main/resources/clojure
@@ -55,6 +55,10 @@ do
help=true
shift
;;
+ --)
+ shift
+ break
+ ;;
*)
break
;;
Code is easier to describe, here's the change I'm proposing ^#2020-01-3016:23alexmillerjust restating to make sure I'm getting it...
clojure takes command lines of the form clojure [dep-opt*] [init-opt*] [main-opt] [arg*] and makes java command lines of the form java [java-opt*] -cp classpath clojure.main [init-opt*] [main-opt] [arg*] and the -- introduces a separator between [dep-opt*] and the rest?#2020-01-3016:24dominicmYes (Although I'm now going to double check that's what happens with my patch)#2020-01-3016:25alexmillernot clear to me how that patch does that, but I get the intent now#2020-01-3016:26dominicm*) delegates to init-opt by catching unrecognized arguments (e.g. -e) and passing them on. -- works similarly, but it does a shift before break in order to remove the -- before continuing.#2020-01-3016:27alexmillerok, I got it#2020-01-3016:27alexmillerseems ok to me if you want to make a ticket/patch#2020-01-3016:27dominicmhaha, I have a silly way to prove that it works.
❯ echo '(println "HELLO")' > -A:kafka
❯ clojure -- -A:kafka
HELLO
#2020-01-3016:27dominicmOkay, sending both in now 🙂#2020-01-3016:27alexmilleralso need to handle: windows script, help text, and man file#2020-01-3016:29dominicmMaybe a stupid question, where's the windows script? Not in brew-install?#2020-01-3016:30borkdudeThis is the one I think: https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/ClojureTools.psm1#2020-01-3016:30dominicmI don't see it on master#2020-01-3016:30alexmillerthat is the correct branch#2020-01-3016:30alexmiller1.10.1#2020-01-3016:31borkdudebtw dominic, Powershell also works on Linux and Mac nowadays, if you want to test some snippets#2020-01-3016:31alexmillerthe change is probably pretty obvious#2020-01-3016:32borkdudeeven then it's nice to test, unless you're not making obvious mistakes all the time like I do 😛#2020-01-3016:32dominicmMy distro hasn't got powershell landed yet, some bug with libressl and dotnet.#2020-01-3016:32alexmillerthe man file is in doc/clojure.1#2020-01-3016:33alexmilleralso, you should make a corresponding PR on https://github.com/clojure/clojure-site with a doc update to content/reference/deps_and_cli#2020-01-3016:34alexmilleranother test to check is that there isn't an issue with the built-in clojure.main support for - arg#2020-01-3016:35alexmillerI think it's fine, but would be good to check it#2020-01-3016:36dominicmThere isn't a problem 🙂#2020-01-3016:41dominicm@alexmiller Just to check, happy with "Stop parsing arguments, and pass remaining arguments to clojure.main" as the doc?#2020-01-3016:52dominicmI've gone ahead with that string, can easily swap it out if you'd prefer something else 🙂#2020-01-3017:02alexmillerI don't know, whatever. I'll update after I apply and look at it#2020-01-3017:04dominicmhaha, okay 🙂#2020-01-3017:17jffryaws-vault uses this pattern (a tool for acquiring temporary credentials in AWS, and then running a command with those creds as env vars)
aws-vault exec profile-name --flag-for-aws-vault -- some-command --flag-for-command#2020-01-3018:37seancorfieldtools.cli supports that too since it's a GNU convention https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html#2020-01-3018:37seancorfield"The argument ‘--’ terminates all options; any following arguments are treated as non-option arguments, even if they begin with a hyphen." (to save folks reading it)#2020-01-3018:51alexmillerthx#2020-01-3018:55borkdude@seancorfield I looked for that on the tools.cli README earlier because of this conversation but Cmd+f on -- didn't get me anything#2020-01-3019:01seancorfield@borkdude Yeah, the README doesn't mention it -- it just links to the GNU doc as the "spec" of the behavior.#2020-01-3019:02borkdudeAh -- thanks 🙂#2020-01-3104:08alexmillertools.deps.alpha 0.8.661 and clj 1.10.1.507 are now available
• New: concurrent downloads make things significantly faster when downloading lots of stuff
• New: -Sthreads can be used to explicitly set the concurrency level, but you generally shouldn't need it
• New: -- can be used in clj as a separator between clj dep args and clojure.main args. For example clj -A:tool -- -A:foo will activate the tool alias, which may specify a main class, then -A:foo will be passed to that program (thanks Dominic Monroe!)
• Fix: regression in -Spom not populating <repositories> since tda 0.8.573
• Fix: manifest type lookup on resolved git dep coordinates (probably of interest to almost no one, but affected tools.deps.graph) #2020-01-3105:06seancorfieldNice job! The concurrent downloads is very cool!#2020-01-3105:07seancorfieldbrew doesn't known about 507 yet it seems... caching?#2020-01-3105:08seancorfieldAh, interesting. It knows about it on macOS but not on Linux yet.#2020-01-3105:16andy.fingerhutNot familiar enough with brew yet, but perhaps brew update will force it to get fresh version info for all packages.#2020-01-3105:24seancorfieldbrew upgrade clojure causes brew to update all its formulae anyway I think... certainly looked like it did.#2020-01-3105:24seancorfieldI suspect the Linux archives lag behind the macOS archives.#2020-01-3105:27seancorfield{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "562533373835161213051d0219067b070364031c6718"}, :content ("[email protected]")}
Ubuntu on WSL1 on Windows 10.#2020-01-3105:30andy.fingerhutProbably. I get same result on Linux vs. macOS right now.#2020-01-3105:33alexmillerLinux always lags#2020-01-3109:47borkdudedeps.clj also lags, but is now updated! https://github.com/borkdude/deps.clj/releases/tag/v0.0.8#2020-01-3110:07vlaaadI haven't really used deps.clj, but I think it's great because it has good windows support — through cmd, not power-shell#2020-01-3110:08vlaaadI wonder how probable it is for clojure/`clj` tools to start using graalvm for cross-platform support#2020-01-3110:09borkdudeif cognitects wants to adopt (parts of) deps.clj, it's free and I'm there if they need help#2020-01-3113:53alexmillerI'm somewhat interested in exploring graal for the classpath construction phase of clj, however, I do not think it makes sense for the official clojure tool to run something other than the jvm on windows - this is how people start their actual Clojure apps, and Graal is not an officially supported platform for Clojure. clj is possibly a different story for interactive repl use, but even then I don't think it makes sense to be running a non-official Clojure interpreter as the repl.#2020-01-3113:55alexmillerfortunately, there's no reason we can't have multiple options!#2020-01-3114:03borkdude@alexmiller I think you're misunderstanding something. deps.clj is only a replacement for the bash part. it's compiled to native using the official clojure and doesn't use a non-official clojure interpreter.#2020-01-3114:03borkdudelike the bash script, it invokes a JVM#2020-01-3114:04borkdudethere also an option to execute the script (non-binary) with babashka, but you don't have to#2020-01-3114:04alexmillerah, so that is different than what I understood#2020-01-3114:05alexmillerI still think it's unlikely Rich would ok using a separate interpreter in this stack, but I could be wrong#2020-01-3114:05borkdudeif you're using the natively compiled deps.exe you are *not* using a separate interpreter#2020-01-3114:06borkdudedeps.clj can run as native (compiled with GraalVM) *or* as an interpreted script (interpreted by babashka which is itself compiled with GraalVM)#2020-01-3114:06alexmillerneither of those are supported Clojure things though#2020-01-3114:06vlaaadYeah, deps.clj is a native windows binary#2020-01-3114:07vlaaadso deps.exe really#2020-01-3114:07borkdudeyeah, that's why I chose the name deps.exe to distinguish from the script deps.clj (which is really just a Clojure source file which you can run on the JVM, or with babashka, it's portable)#2020-01-3114:08borkdudebut I get the message and I'm not trying to sell it to you 😉#2020-01-3114:08alexmillergotcha, the Graal path is something that could potentially be added, I guess I didn't realize what you were doing there#2020-01-3114:08vlaaadit's main value over powershell-based runner is that it can run on default windows command line. We explored moving Defold to tools-deps from lein, but other tooling depends on cmd and can't run in pwsh#2020-01-3114:09borkdudethe benefit of deps.exe (or deps.clj as source file) is that you don't have to maintain bash and/or PS#2020-01-3114:09alexmillerso in the graal case, what exactly is compiled there?#2020-01-3114:09borkdudebut otherwise it's exactly the same thing behaviorally#2020-01-3114:10alexmilleris it launching a jvm to do the classpath construction like clj?#2020-01-3114:11borkdude@alexmiller clojure, AOT-ed to .class files which are then translated to a native binary by GraalVM. yes, it invokes a JVM:
https://github.com/borkdude/deps.clj/blob/e6561ac7997bf1fc0f3ee8406e7af7bc09ca3adb/src/borkdude/deps.clj#L511#2020-01-3114:12alexmillerwell, that's cool, I didn't realize all that#2020-01-3114:12borkdude@alexmiller to simplify it conceptually, think of it as invoking the .jar file here:
https://github.com/borkdude/deps.clj/releases/tag/v0.0.8
but with faster startup#2020-01-3114:12alexmilleryeah#2020-01-3114:12alexmillerwell, I'll put it in the thinker then#2020-01-3114:13alexmillerneed to spend some time working on clojure instead!#2020-01-3114:19alexmillerthe potentially interesting variant here (much bigger lift) is to combine the deps.clj stuff with the classpath construction and graal all of that (but I'm sure there are a lot more challenges to do that)#2020-01-3114:30vlaaadknowing @borkdude I would expect a release announcement tomorrow 😆#2020-01-3114:30borkdude@alexmiller I added some development instructions here now: https://github.com/borkdude/deps.clj#developing-depsclj
In case you want to fiddle around 🙂#2020-01-3114:31borkdudeyeah, I tried to graal the tools jar as well, but got into some weird stuff which I didn't explore in much detail#2020-01-3114:33borkdudeI think it was related to the jgit library, but I only remember vaguely#2020-02-0121:12lvhIs there a way to express Maven plugins in deps.edn so that -Spom will include the plugin (and a repo) or do I want to do that by editing the resulting XML#2020-02-0121:13lvhuse case: I want to mess with graal + javafx, specifically gluon substrate, and the recommended way to build stuff there is https://github.com/gluonhq/client-maven-plugin#2020-02-0121:17borkdudeAnother way would be to use clojure.data.xml in a script to manipulate the generated pom afterwards#2020-02-0121:17lvhyep that's what I had in mind with "editing"#2020-02-0121:18lvhI take it clojure.data.xml works great with babashka/sci? 🙂#2020-02-0121:20borkdudeI have an issue about that. It kinda works: https://github.com/borkdude/babashka/issues/153
But it's a little hacky right now. I'm holding off until GraalVM adds proper support for javax.xml.
https://github.com/oracle/graal/issues/1387#2020-02-0121:21borkdudeI'd just use a normal clojure script for it right now.#2020-02-0121:21lvhright, easy enough to port later#2020-02-0122:23alexmiller-Spom will sync into an existing pom, so just put what you want in the pom#2020-02-0122:23alexmillerIt should stay there#2020-02-0212:37borkdudecreated a little script to easily generate git url + latest sha on the command line which you can then paste into deps.edn:
https://github.com/borkdude/babashka/#generate-depsedn-entry-for-a-gitlib#2020-02-0212:39borkdudeto use the script with clojure (JVM) instead of bb, you'll need to change the shebang into #!/usr/bin/env clojure, add a prn around the returning expression and add (shutdown-agents) at the bottom#2020-02-0410:47holyjakHow do you folks search for tools that build on tools deps, such as clj-new and depstar? And/or is there a list of these tools somewhere?#2020-02-0410:50borkdude@holyjak this is documented on the wiki#2020-02-0411:24holyjakThank you. I have sent a PR to the Deps reference to make it easier to find the page :-)#2020-02-0415:28Derek PassenWhat’s the recommended way of mixing in a dev resources directory? I have a :dev alias that has :extra-paths ["dev"] but I can’t call ( "local.edn") assuming that file lives within ./dev/resources.#2020-02-0415:31alexmillerinclude it in :extra-paths in an alias#2020-02-0415:31Derek Passen“dev/resources” ?#2020-02-0415:32alexmillerwell, whatever the path is#2020-02-0415:32Derek PassenI already have “dev” as an extra-path#2020-02-0415:32alexmillersorry, re-reading, yes#2020-02-0415:32Derek Passenthanks alex, I will give that a try#2020-02-0415:32alexmillerthe paths are classpath roots, so any file under that path has to be relative to that root#2020-02-0415:33alexmillerso either dev and resources/local.edn or dev/resources and local.edn#2020-02-0415:33Derek Passenmakes sense#2020-02-0709:47robert-stuttafordhow do folks document the reason for a dep in their deps.edn? i was thinking we could just add a :doc "For feature x" key to the map alongside :mvn/version et al#2020-02-0709:50vlaaad;; need this lib because x#2020-02-0710:14danierouxI have this:
org.clojure/tools.reader {:mvn/version "1.0.0-beta4" :dictated-by :datomic}#2020-02-0710:19robert-stuttafordi like that! what do you do for deps that are for specific feature code, rather than to support another dep?#2020-02-0710:26dominicmI don't!#2020-02-0710:27robert-stuttafordseparate question: is there a dependency conflict diagnosis output in the tools.deps world, like the one lein has when you lein deps :tree?#2020-02-0710:27dominicmI think there's a ticket for this#2020-02-0710:29robert-stuttafordok cool - this is probably the very last thing we'll still need lein for!#2020-02-0710:30dominicmLein handles conflicts very differently#2020-02-0710:30dominicmI wouldn't compare the outputs at all#2020-02-0710:31delaguardothere is a way to generate tracefile with https://github.com/clojure/tools.deps.graph
right now there is no tooling to work with those traces and you have to analyze it yourself but good to know as an alternative#2020-02-0710:31dominicmPersonally, I've stopped having problems when using deps.edn. It has better conflict resolution behaviour.#2020-02-0710:32dominicmYeah ^^ trace file has the data.#2020-02-0710:32robert-stuttafordfantastic#2020-02-0710:27dominicmYou're creating 2 things you need to sync up. You won't remember when you need a new feature.#2020-02-0710:28robert-stuttafordi agree, @dominicm -- the counter argument though is when upgrading dependencies, it's a lot of effort to dig around and understand how they are used. so i'm not planning reams of content. just enough to understand its importance and connection#2020-02-0710:29dominicmOn my list is the job to solve that through tooling :)#2020-02-0710:30dominicmKondo should make it fairly trivial to do at the namespace level#2020-02-0710:30robert-stuttafordgreat point!#2020-02-0710:31borkdudehere's an example to do some namespace analysis stuff:
https://github.com/borkdude/clj-kondo/tree/master/analysis#circular-dependencies#2020-02-0710:32robert-stuttafordyou're like Varys from Game of Thrones, with little birds telling you when people talk about kondo 😄#2020-02-0710:32borkdudeI will now crawl back under my rock#2020-02-0710:33robert-stuttafordhahah i think it's great!#2020-02-0710:32borkdudehere's another one doing a namespace graph:
https://github.com/borkdude/clj-kondo/tree/master/analysis#namespace-graph#2020-02-0710:32dominicmThe trick is figuring out which jars have which namespaces defined within, then mapping that to the dependency#2020-02-0710:33dominicmKondo supports jars though, so it isn't too difficult.#2020-02-0710:34borkdudekondo also reports in which files the namepace are defined. it also lists the jar when you use the option {:output {:canonical-paths true}}#2020-02-0722:25dominicmMight be better to not use kondo actually, as then I'll miss datomic and other AOT libraries#2020-02-0710:39borkdudethe jar string is a little bit weird though, I want to make it more compatible with https://java.net, so it might change a little#2020-02-0716:30dominicmIt should probably be the same as what io/resource would give me#2020-02-0717:06borkdudeyeah, that's what I want to make it#2020-02-0717:07borkdudeso expect a small breaking change there#2020-02-0717:09dominicmOkay, won't write my code this weekend then.#2020-02-0717:09borkdudeit's only a small change, you can still use it to compare if it came from a different jar, to e.g. discover namespace conflicts#2020-02-0717:10borkdudejar:foo:bar will become something like <jar://file>://foo#!/dude or something, I can't remember these details#2020-02-0717:10dominicmI do that with sed atm. I think you really want to compare jars for conflicts at the file level.#2020-02-0717:11borkdudemaybe you can just specify what you need, so I can take that into account#2020-02-0717:11borkdudeif something isn't working for that purpose right now#2020-02-0717:16dominicmI could make what's there work, don't worry. I'm just patient enough. http://Java.net compatibility would be good.
What I'm planning on doing is linking the jar back to its dependency (eg bidi/bidi), then I know which namespaces use which dependency.#2020-02-0721:51souenzzoI just realized that {} is a enough deps.edn for some libraries#2020-02-0721:54alexmillerthat was the idea :)#2020-02-0721:54alexmillerif you have no deps but Clojure and typical src dir, you should be good :)#2020-02-0721:56borkdudeyep, it's awesome. and even if a library (like medley before not too long ago) doesn't have one, you can declare a manifest type#2020-02-0722:41seancorfield@souenzzo or even no deps.edn at all 🙂
(! 1222)-> mkdir -p example/src/simple
(! 1223)-> echo '(ns simple.main) (defn -main [& args] (println "Hello, World!"))' > example/src/simple/main.clj
(! 1224)-> cd example/
(! 1225)-> ls
src
(! 1226)-> clojure -m simple.main
Hello, World!
#2020-02-0722:42souenzzoBut without a file with a empty map you cant import as a git-dep#2020-02-0722:43seancorfieldYes you can. You can specify a manifest type of :deps#2020-02-0722:43borkdudeYou can:
Local project coordinate: {:local/root "/path/to/project"}
Optional key :deps/manifest
Specifies the project manifest type
Default is to auto-detect the project type (currently either :deps or :pom)#2020-02-0722:43borkdude(https://clojure.org/reference/deps_and_cli)#2020-02-0722:44seancorfieldDang, you're fast! I was just about to paste that in! 🙂#2020-02-0722:44borkdudeIt was still in my clipboard from the previous time someone asked (kidding)#2020-02-0722:47seancorfieldFor example
(! 1234)-> clojure -Sdeps '{:deps {jedi-time {:git/url "" :sha "e3c9ab41267be7d9033a13e1d666a3411f53d3c6"}}}'
Cloning:
...
Error building classpath. Manifest type not detected when finding deps for jedi-time/jedi-time in coordinate {:git/url "", :sha "e3c9ab41267be7d9033a13e1d666a3411f53d3c6"}
(! 1235)-> clojure -Sdeps '{:deps {jedi-time {:git/url "" :sha "e3c9ab41267be7d9033a13e1d666a3411f53d3c6" :deps/manifest :deps}}}'
Clojure 1.10.1
user=> #2020-02-0722:45borkdudeWould it make sense for tools.deps to default manifest type to deps.edn if a project has no pom.xml / deps.edn? Why not?#2020-02-0722:46ghadiif there's no manifest, why default the manifest type?#2020-02-0722:47ghadithere's an implication/assumption I don't get#2020-02-0722:48borkdudein the case of git deps with only a src folder, that saves you one file or some config 😛 probably a stupid reason#2020-02-0722:48ghadisounds stupid (you said it 🙂 )#2020-02-0722:48seancorfieldI think defaulting to :deps would lead to confusing errors when someone tries to mistakenly depend on a Leiningen project that had other dependencies.#2020-02-0722:49ghadigetting a magic ambient default seems like an undesired thing...#2020-02-0722:50souenzzothere is only one default:
paths ["src"]#2020-02-0722:50borkdudethe reason I brought it up is that this option is often not known to people, maybe because the guide isn't very elaborate about it, although it does document it#2020-02-0722:52andy.fingerhutIf http://Clojure.org folks are open to a PR enhancing the docs, and you are open to writing such an enhancement, I see magic occurring in the near future 🙂#2020-02-0722:53ghadiI guess I don’t understand the problem enough to evaluate a solution #2020-02-0722:53ghadiIf you control the dir you can add a manifest easily#2020-02-0722:55borkdudeI was thinking the other way around: sometimes it's desirable to depend on a clojure project out of your control that only has a project.clj but no pom and no dependencies.
what I did for medley is add a deps.edn to it: https://github.com/weavejester/medley/commits/master
because that was more convenient to me than having to look up the exact option for manifest type again 😃#2020-02-0722:56borkdudeanyway, not that important I guess. have a good weekend y'all#2020-02-0722:56ghadiIt’s been a while since I tried but I though project.clj’s were read automatically #2020-02-0722:56ghadiSee ya @borkdude #2020-02-0722:57andy.fingerhutI don't think project.clj files are read if you depend upon a git commit SHA of a Leiningen library that has no deps.edn, is it?#2020-02-0722:57andy.fingerhutmaybe tools.deps has a way to do that I do not realize, but it seems like that would be kind of magical ambient default, wouldn't it?#2020-02-0723:00andy.fingerhutI could be way off here, but my guess was that a Leiningen library deployed as a JAR, e.g. to http://clojars.org, typically contains a pom.xml that tools.deps would use.#2020-02-0723:00alexmillerproject.clj's are not read now#2020-02-0723:00alexmillerbut maybe they will be in the future#2020-02-0723:01alexmillerI don't really want to change the defaults until that's sorted#2020-02-0723:01andy.fingerhutI'm sure you know that project.clj files can contain arbitrary executable code...#2020-02-0723:03borkdudeso do sources files 😉#2020-02-0723:03andy.fingerhutdeps.edn files don't, though.#2020-02-0723:03andy.fingerhutnor do pom.xml files#2020-02-0723:05alexmillerless true than you think#2020-02-0723:06alexmillerpom model resolution is remarkably f'ing complicated#2020-02-0723:06alexmillerand the libs around it, more so#2020-02-0723:07andy.fingerhutOK, news to me, but I am pretty sure you have looked at it 1000x longer than I have 🙂 I was assuming that pom.xml didn't have executable / evaluatable pieces to them.#2020-02-0723:08alexmillerto actually resolve pom files the way mvn does, you need to talk to the web, download plugins + parent poms, etc#2020-02-0723:09alexmillerand there are a lot of properties that need to be interpolated as well#2020-02-0723:10alexmillertools.deps is tapping into apis to do the right thing for most of that (and most people don't do anything very fancy)#2020-02-0723:04borkdudemaybe more people should get into the habit of checking in the generated pom.xml files if they want their non deps.edn libs to be used#2020-02-0723:05andy.fingerhutFYI, I am not here asking for new tools.deps features -- just checking whether I am missing something in what it does now.#2020-02-0723:06alexmillerI actually think it would be good to have project.clj support#2020-02-0723:07alexmillerI am undecided whether it should be naive support (data only) that relies on no leiningen stuff or good (perhaps add-on) support that does the actually right thing#2020-02-0723:07alexmilleror both#2020-02-0723:08borkdudenaive as in only read the :dependencies vector and be done with it?#2020-02-0723:10andy.fingerhutI would guesstimate that over 90% of project.clj files start with (defproject and do not rely on code execution for the value of their top level :dependencies key, but haven't done any kind of wide survey.#2020-02-0723:09borkdudewould cover the first 80% probably#2020-02-0723:11alexmilleryeah. honestly I would love to know the actual number.#2020-02-0723:11borkdudesandboxed execution of project.clj could also work for the remaining 20%#2020-02-0723:12alexmilleryou mean eval?#2020-02-0723:14borkdudeyes. personally I would think it would be ok to only accept EDN-parsable project.cljs and ignore those with read-evals.#2020-02-0723:14andy.fingerhutI am sure developers often download source of a Leiningen project and run Lein commands before looking whether project.clj does code execution, but at least they had to download the source for that to happen. Doing eval on the project.clj of transitive dependencies is another level of openness to exploits.#2020-02-0723:14alexmillerthere's a fairly easy path to do the right thing using the lein code base iirc, but it does require depending on that code (which pulls in a bunch of stuff)#2020-02-0723:15alexmillerdoing this the really really right way (like eval in clojail or something) is probably not something I personally am ever going to write :)#2020-02-0723:15borkdudethat's why I said "sandboxed", something like clojail or sci, so allow only a restricted set of functions. it sounds like a bit of a headache that may not be worth it for the 20% or less#2020-02-0723:15andy.fingerhutthink of the endless bug reports ...#2020-02-0723:15borkdudesame thoughts there#2020-02-0723:16alexmillerlike I said, I'd be interested in knowing a better estimate of the %#2020-02-0723:16alexmillerif it's 99% then that's an easier decision#2020-02-0723:16borkdudemaybe it's more like 5% or 1%#2020-02-0723:16andy.fingerhutIs the % number you are interested in mainly "is pure edn" vs. not? Or some finer distinctions would be interesting to know?#2020-02-0723:17borkdudeI guess you can retrieve a list of all clojars projects and then curl to their github repo and try to parse it with edn/read-string#2020-02-0723:18borkdudeI have swapped from read-eval to just templating an EDN-readable project.clj recently. the templating is done with clojure ironically 🙂#2020-02-0723:18alexmillerjust whether a naive slurp edn/read find-:dependencies would work#2020-02-0723:19alexmillerif anyone wanted to get a believable estimate and write that code, I'm happy to consider it#2020-02-0723:19andy.fingerhutand likely ignore any :profiles with their own :dependencies keys#2020-02-0723:20alexmillernot doing anything down that line#2020-02-0723:21alexmillergotta run... later#2020-02-0723:21borkdudejust so we're clear, this is about using project.clj projects as git deps right?#2020-02-0723:21borkdudethat would probably exclude projects that have java sources in them#2020-02-0723:21andy.fingerhut@borkdude I am happy to work on my other hobbies if this has set a fire under you. You would probably beat me to the answer, anyway 🙂 One perhaps additional interesting thing to check, if it isn't difficult, is whether the http://clojars.org pom.xml is "equivalent" to the top level :dependencies key.#2020-02-0723:22borkdude@andy.fingerhut feel free to give this a go, I'm not doing this anytime soon, got lots of other stuff to do atm#2020-02-0723:25andy.fingerhutSomehow reminds me of a scene from an old Simpson's show: https://www.youtube.com/watch?v=6qUQ94_MF2Y#2020-02-0723:26borkdudeI didn't even ask for project.clj support btw, hehe#2020-02-0723:27borkdudeAlthough I would not be against it#2020-02-0723:28andy.fingerhutThat's the beauty of it -- neither of us did, but Alex was already wondering about doing it 🙂#2020-02-0723:29borkdude😃#2020-02-0723:23andy.fingerhutAs a sanity check on whether people often deploy pom.xml files that are a different dependencies list than the top level :dependencies key#2020-02-0723:26borkdudeUsually when I need a project.clj as git dep really bad, I just fork it and add the deps.edn myself#2020-02-0723:32borkdudeNot directly related to this problem, but there is a repo which collects stats from clojars as edn files:
https://github.com/plexus/clojars-stats
Could be of use for some kinds of analyses#2020-02-0723:35andy.fingerhutThanks for pointing that out. If nothing else, it might be a good way to get a large list of projects, and sort them by download popularity or something like that.#2020-02-0723:35andy.fingerhutand drive a program that downloaded more stuff about each project found there.#2020-02-0819:49mssis there a way to specify a path to a specific deps.edn file when using clj?#2020-02-0819:52alexmillerno#2020-02-0819:52alexmillerwhat's the use case?#2020-02-0820:27msswriting a bash script in one dir to build a module at another path#2020-02-0820:27mssobv easy to work around, was just wondering if I was missing something obvious. thanks!#2020-02-0820:41Nate SuttonI have multiple single-file projects in the same directory each with their own deps file. I have to read the deps file in and then use the contents as an argument in order to work around not having a way to specify a deps file#2020-02-0820:42Nate Suttonso, that's a use case#2020-02-0820:50borkdude@mss fwiw, deps.clj does support this with the non-standard -Sdeps-file option#2020-02-0821:13seancorfield@mss If you do CLJ_CONFIG=/path/to/other/folder cllj it will look there for the user-level deps.edn (instead of you home directory) which might give you part of the solution?#2020-02-0821:19seancorfieldI like the -Sdeps-file idea tho'. I would use that quite often I suspect if clojure supported it. @alexmiller would that even be considered as an enhancement?#2020-02-0821:22borkdudeI often use it when I need the classpath from some other project in some other directory to lint with clj-kondo#2020-02-0821:23borkdudealthough $ ( cd /tmp/project ; clojure -Spath ) works too#2020-02-0821:24seancorfieldI suppose you can always use clj -Sdeps "$(cat /path/to/deps.edn)"#2020-02-0821:26borkdudethat works, but your local deps.edn will still be merged in, which is not always what I want#2020-02-0821:26borkdude-Sdeps-file will regard that file instead of the local deps.edn#2020-02-0821:28seancorfieldYeah, there are a lot of possibilities around which combination of files you might want to work from.#2020-02-0821:55alexmillerI've walked down the path of adding this a couple times for different reasons but there are a lot of unobvious consequences#2020-02-0821:55alexmillerlike do you interpret relative paths in deps.edn from the deps.edn location or current location?#2020-02-0821:56alexmillerthere are also now a lot of code locations that assume the deps.edn name, not just in tools.deps.alpha but in external tools around it#2020-02-0821:58alexmillerat the moment the tradeoffs don't seem worth it to me but I am interested to hear about cases where people need stuff like this though so I can continue to evaluate that#2020-02-0900:58Nate SuttonI think the relative path question could be solved by splitting the concepts of which deps file to use from the location of the project root#2020-02-0900:59Nate SuttonI agree it expands the scope, though#2020-02-0901:02Nate Suttonit's alpha, thankfully, so changes like this would be ok?#2020-02-0901:23andy.fingerhutI suspect Alex is in a mind set where even though it is alpha, adding things that he doesn't see value in supporting long term is like intentionally planning to pull the rug out later.#2020-02-0903:22alexmillerit's not alpha in my mind#2020-02-0903:22alexmiller;)#2020-02-0903:23alexmillerRich and I just haven't had the time to think through what changing it to be non-alpha means yet#2020-02-0906:54Nate Suttonit's not alpha??#2020-02-0907:51andy.fingerhutIf I interpret what Alex said correctly, it is currently alpha, but he and Rich want it to become non-alpha soon-ish.#2020-02-0907:56alexmillerit has alpha in the name, but I have been maintaining it like it's not alpha for a long time, and believe the api is stable#2020-02-0908:08borkdudedeps.clj non-standard features are certainly alpha though, I think it can be a place of experimentation as far as the bash part goes#2020-02-0910:07peterwestmacottMorning! I've just got an error from my IDE saying: "The following errors were found during project resolve: /home/peter/personal/mandel-zero/deps.edn: Destination path "see" already exists and is not an empty directory" - it doesn't tell me where this non-empty directory is though! Is there a place where tools-deps caches dependencies from github that I can go and delete things from?#2020-02-0910:14dharriganHave you tried to delete .cpcache?#2020-02-0910:16peterwestmacottI think its ~/.gitlibs#2020-02-0910:28peterwestmacottoh, dear, I seem to have broken it. it now just says 'Auth fail' instead#2020-02-0914:59alexmillerwhat happens if you are just at the command line?#2020-02-0915:00alexmillerthese sound like ide specific issues, not tools.deps#2020-02-0915:20peterwestmacottstrangely not. The ide appears to be reporting an underlying error#2020-02-0915:21peterwestmacotton the plus side I do get a stack trace though#2020-02-0915:21peterwestmacottCloning:
#2020-02-0915:24peterwestmacotttbf I did go deleting directories in ~/.gitlibs to try and fix an earlier problem, which was probably a little reckless - I was sort of expecting them to get recreated as needed#2020-02-0915:48alexmillerwhat git url are you using?#2020-02-0915:52alexmillerI guess something like /cdn-cgi/l/email-protection:peterwestmacott/draw.git#2020-02-0915:52alexmillerwhat os are you on?#2020-02-0915:54alexmilleris your ssh-agent running? does it have your ssh identity?#2020-02-0915:54alexmillerssh-add -l should list your key#2020-02-0916:26peterwestmacottThe agent has no identities.#2020-02-0916:26peterwestmacottI'm guessing that's bad#2020-02-0916:26peterwestmacottI'm running xubuntu#2020-02-0916:28peterwestmacottoh, that seemed to sort it out. THANKS ALEX!!!!#2020-02-1010:11andy.fingerhutIs there a way in one's $HOME/.clojure/deps.edn file to have a :local/root dependency with a relative path name starting from your home directory? I tried {:local/root "$HOME/path/to/my.jar"} and {:local/root "~/path/to/my.jar"} , but neither seemed to work#2020-02-1010:11andy.fingerhutI ask because it would be helpful to have such a thing for sharing a ~/.clojure/deps.edn file contents across multiple systems, where my home directory absolute path name differs between them.#2020-02-1010:14borkdudetildes are not expanded in .edn files#2020-02-1010:15borkdudeneither is $HOME#2020-02-1010:16borkdudebut I get the problem, would be nice to see a solution for#2020-02-1010:56sogaiumy current work-around is to put all source under ~/src/ (or some other directory right under my home directory) and make paths relative: e.g. "../REBL/REBL.jar"
ofc, that means on each system source code needs to live under a directory right under ~.
on a related note, to get the effect of "variables" in deps.edn, i've been thinking whether to generate deps.edn (possibly from some template).#2020-02-1010:57borkdudewith some lightweight scripting / templating you can get around it though#2020-02-1010:58borkdudeah, you also suggested that sogaiu 😉#2020-02-1013:45alexmillerI'm going to continue to resist variables in deps.edn for as long as possible :)#2020-02-1017:52andy.fingerhutIs tilde as alias for home directory in same category?#2020-02-1017:52andy.fingerhutNot an urgent need by any means, just asking while we are in the neighborhood.#2020-02-1017:52alexmilleryes#2020-02-1017:52alexmillerit's in the category of "processing step beyond reading as data"#2020-02-1017:53alexmillerhaving none of those is a huge advantage for downstream work#2020-02-1020:47mike_ananevJust hacking my own build tool right now, and I created new file build.edn and code for processing with comment in code
"This sections dropped to build.edn cause Alex Miller continue to resist variables in deps.edn"
😎#2020-02-1020:50andy.fingerhutI mean, you can give that as the reason if you want, but as an engineer, I'd say addressing the reasons for that design decision, rather than focusing on the person that made them, is a good practice. Sure, include the name of one or more people responsible for making the decisions if you want, but without the reasons, it can sound like you believe the decisions are arbitrary, or not grounded in any kinds of reasons. Not saying you have done that -- just pointing out the possibilities of misinterpretation.#2020-02-1014:03dominicmI think generating your own deps.edn is a good idea#2020-02-1016:08sparkofreasonAre there tools/methodologies around adapting lein projects containing java to be amenable to clj/`deps.edn` ? I'm looking at a potential dependency that is fairly new and fast-moving, would like to be able to reliably pin my reference to a known SHA, so I can reference the code at that point rather than depending on potentially out-of-date docs, etc. I could manually run lein and deploy it locally, but something more automated where I can just update the SHA and have clj do the right thing would be nice-to-have.#2020-02-1016:15borkdude@dave.dixon The most reliable way I know is fork it and add a deps.edn to it.#2020-02-1016:17sparkofreasonYes. But then how do I get the downstream project to compile the java and put the classes on the classpath? I think I could do it for the specific case of that project using an alias and a tool like badigeon, but curious if anybody had come up with anything more general.#2020-02-1016:18borkdudeif it contains java, it's a bit of a different story. I would just deploy it as a leiningen artifact to your own org on clojars in this case#2020-02-1016:19borkdude(my personal take on it, for compiling java I always use lein, since it just works)#2020-02-1018:32sparkofreasonSort of hacked it by building the java classes, and adding that directory to the :paths . Pushed to github, and that works. Curious if there's any reason to not do this.#2020-02-1109:44rickmoynihan> Curious if there’s any reason to not do this.
It’s not good to commit complied (and binary) artefacts into source control.
It would be better to compile the java classes and deploy it as a library. Either a shell script with javac and some mvn calls, or just move the java code into its own lein project and use lein & project.clj to compile and deploy the java bit, and depend on it from the tools.deps lib.
Whether to break out the java as a separate lib largely depends on how stable it is, and how often you expect to change it etc.#2020-02-1114:36sparkofreason> It’s not good to commit complied (and binary) artefacts into source control.
I've heard this, and my understanding (maybe wrong) for the rationale is that git doesn't handle large binaries well, as might be the case for a jar or fully linked executable. Class files, however, are pretty small, and map closely to the corresponding java source.#2020-02-1114:43sparkofreasonThe key risk is having the compiled class files be out of sync with the source. But that's kind of the problem I'm facing anyway, source is changing, and the clojars artifacts may not match, which makes it hard to figure out which features are there etc. More ideal would be a tool that built the java classes after clj pulled from the indicated :sha.#2020-02-1114:56rickmoynihanWell there are a whole bunch of reasons:
1. git doesn’t handle large files in a repo very well (operations on git repos slow to a crawl on large repo’s, e.g. changing branches etc)
2. you can’t diff a binary very well, so merging branches etc becomes painful. Should you accept a change or not?
3. you have a dependency between source and output files in the repo
4. related to 3, but for non-binary output files; e.g. compiling and minifying css/js you get huge diffs in your repo that can cause some tools to crash/struggle to render. You can set attributes on the files in git, but not all tools honor them properly.
5. different versions of javac generate different byte code for the same source code so you may get diffs and deltas just by upgrading java versions
6. tools.deps can’t handle files in a repo over ~100mb (due to limitations in jgit)
7. adopting this practice on repo early on, normalises the behaviour, and all sorts of binaries get commited; git performance tanks, and then you have to rewrite the whole history with filter-branch, or worse because of lack of patience around rewriting the history correctly the team decide to drop all the git history and start fresh in a new repo.
I’ve worked in many repo’s where this has been done, and it’s almost always caused problems. Not saying you won’t get away it; just be aware of the problems.#2020-02-1114:57rickmoynihanI’ve also personally experienced all of those at some point in the last 15+ years or so using git.#2020-02-1116:29sparkofreasonAgreed in general. And most of this sounds like something that should be fixed in git, making the abstraction less leaky, but that's a whole other thread. Clearly it would be better to have a way to trigger the java build after clj pulls and update the classpath accordingly.#2020-02-1116:39rickmoynihanThere are fixes in/around git for things like large-files, and git handles binaries ok… However you can’t really solve the issue of how to merge a random binary; it’s essentially pick one or the other or commit a new one. I’m not sure git can really do much more to help there.
For me philosophically it’s just the wrong thing to do; git is for source control and generated files aren’t source. I’m saying this as someone who has committed binaries into repos though; and may at some point do it again… I’d just always very much prefer to avoid it.#2020-02-1017:56andy.fingerhutI have not been working on this full time by any means, but from a programmatically generated file from http://Clojars.org, I have a list of 18,493 projects and their URLs they list in the artifact there, which appear to be on Github, downloading now. This is partial progress towards answering the question "what fraction of Leiningen project.clj files are readable using clojure.edn/read".#2020-02-1017:57andy.fingerhutAs an aside on the kinds of weird things I now expect more often in such large-ish data sets: 672 projects with the incorrect URL ssh-keygen and adding it to the openssh agent via ssh-add.
I think the relevant part of the stacktrace is:
Caused by: org.eclipse.jgit.errors.TransportException: : Authentication is required but no CredentialsProvider has been registered
#2020-02-1020:44samIs there something I’m missing? I was able to get it working fine on my mac.#2020-02-1020:44dominicmThat's https, you need to use ssh @samwagg0583#2020-02-1020:45dominicmChange the url to be the ssh one on github#2020-02-1020:45samoh wow, I can’t believer I did that and have been spinning my wheels for hours!#2020-02-1020:51samThat did it, @dominicm. I’m embarrased, but thanks so much.#2020-02-1022:12hlollione nix package (Lumo) suddenly started to fail with
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.clojure:clojure:pom:1.10.1 from/to central (): : Name or service not known
I'm using tools-deps to resolve the dependencies, but I highly doubt this is related to tools-deps. But could be some mirror setting I guess? This at least isn't network error since Hydra (the CI of nixpkgs) also reported this exact same error. Maybe this error sounds familiar to someone?
Full stack here 4wiw https://pastebin.com/sC6d0Cdn#2020-02-1022:15andy.fingerhutIn a web browser, I can go to
successfully. Maybe some transient failure, or networking problem between that host and public Internet?#2020-02-1022:17ghadiyou need to paste your settings for fetching these deps#2020-02-1022:18ghadior whatever nixpkgs is doing#2020-02-1022:18hlollithis was also reported to me here
https://gist.github.com/Mic92/e7ac6eb8652b72ddc9e5a349b8e7bc23#2020-02-1022:18ghadiseems pretty clear:
Caused by: https://java.net.UnknownHostException: https://repo1.maven.org: Name or service not known
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)#2020-02-1022:19ghadieither transient name resolution failure, or JVM is mistakenly resolving maven through IPv6 instead of IPv4#2020-02-1022:19hlollilet repos = [
""
""
""
""
""
];
....
{
name = "org.clojure/clojure";
path = pkgs.fetchMavenArtifact {
inherit repos;
artifactId = "clojure";
groupId = "org.clojure";
sha512 = "f28178179483531862afae13e246386f8fda081afa523d3c4ea3a083ab607d23575d38ecb9ec0ee7f4d65cbe39a119f680e6de4669bc9cf593aa92be0c61562b";
version = "1.10.1";
};
}
...
so I download the jars then call clojure basically#2020-02-1022:20ghadialso the error you're pasting is a different version than the one in nix#2020-02-1022:21ghadi1.10.1 vs 1.10.0-beta5#2020-02-1022:21hlollithe latter one uses beta, yes, same error still. I tried bumping.#2020-02-1022:21ghadibut still, the error cause is accurate#2020-02-1022:21ghadiit has nothing to do with the version#2020-02-1022:21ghadithere's a name resolution error#2020-02-1022:21hlolliyes, something in that direction was my instinct#2020-02-1022:22hlollijust dns lookup?#2020-02-1022:22ghadiinstinct / gut feeling unnecessary 🙂
you just have to read the stacktrace#2020-02-1022:22ghadiit appears like it's resolving through IPv6#2020-02-1022:22andy.fingerhutWell, or it is using IPv6 when IPv6 is disabled somewhere in the network between you and DNS resolver, or between your host and where JAR file is.#2020-02-1022:22ghadinot sure if the JVM always appears like it's resolving through IPv6#2020-02-1022:22hlolliwell, I was only explaining why the first paste was in diferent version, but yeh, nevermind#2020-02-1022:23ghadiyou can try to set -.preferIPv4Stack=true on the JVM to see if it resolves differently#2020-02-1022:23ghadior try to resolve something that you know is invalid, and see if you get the same trace#2020-02-1022:24hlollinice, that sounds like something I can try. I'm using 8.8.8.8 like always. Maybe maven isn't discoverable there (for me dns lookups are black boxes)#2020-02-1022:24hlollithanks for the tip fellas 🙂#2020-02-1022:25ghadinp. you should test your hypothesis#2020-02-1022:25ghadialso check if nix changed their global ipv6 settings or something#2020-02-1022:26hlollithat's also plausible, tough it could only be the package manager that could have such power, all else is configured by the user#2020-02-1022:27hlolliit's a beginning of a bug hunt I guess, again, thanks#2020-02-1022:28andy.fingerhutSounds like it. We can throw the hot potato into what looks like the most likely hands for the problem, which at least in this case strongly appears not to be deps.edn code, nor any other particular package manager type of thing.#2020-02-1022:31alexmillerI was on a thread last week about some Maven deploy errors (different problem) and it sounded like they've been doing some server updates. could be something on their side. They have a jira system for reporting issues if you wanted to ask there. https://issues.sonatype.org/secure/Dashboard.jspa#2020-02-1022:34ghadias a first step you should try to figure out what DNS settings you have in /etc/resolve* and what address types you have set with ip addr show#2020-02-1023:04hlollireddest herring this week, I actually had stable and good connection to maven, the error was arriving at installation time and not build time, which in nix mean no unwanted side-effect (calling http(s)). So to take my quesiton to a new direction, why would the clojure executeable try to download clojure-1.10.1 when I specify its jar location on the classpath?
My classpath is (stored as classp)
/nix/store/58c2jsc0gq4mi73ma6pvphgnvnpwlnn8-com_cognitect_transit-java-0.8.332.jar:/nix/store/bayaxp12xw9kpbxn35pwab2s4ya8gpcn-org_clojure_data_json-0.2.6.jar:/nix/store/z26fq48r5xs7syp1j2df5pjv1qk37pjf-org_clojure_clojure-1.10.1.jar
...elided...
and command is
${clojure}/bin/clojure -Scp ${classp} -e "${compileClojurescript true}"#2020-02-1023:08hlolliwell, I'll open a ticket if I find a clear regression between previous versions. No need to solve it here now.#2020-02-1023:08andy.fingerhutAs a guess (I do not know implementation details), but there might be cases where clojure finds a different version in local file system, and time stamps or I-don't-yet-know-what cause it to make network connections to see if a newer version isa available?#2020-02-1023:09hlollithat could be, for security resons, all file timstamps are set to 0 (1jan1970).#2020-02-1023:10andy.fingerhutIf you want to absolutely prevent this possibility, you could use clojure command to create a classpath as its output, and save that in a one-line bash script#2020-02-1023:10andy.fingerhutUsing -Spath option.#2020-02-1023:11andy.fingerhutbefore you reach installation time, or whatever step doesn't like you to make network connections.#2020-02-1023:11hlollihmm, good point#2020-02-1023:12hlollithat should be easy thing to test. The maven endpoints are initially made this way. I make tools-deps create classpath and maven endpoints which I have nix download for me, and then provide it again to clojure. Has worked fine last 2 years.#2020-02-1023:13hlollithen the question is if the generator and the build environment are mismatching, tough I updated 2 months ago, there's a chance that nixpkgs has even newer clojure data.#2020-02-1023:13andy.fingerhutSomething changed, but I don't know what. Depending upon how much time you like tracking down such root causes, and how important it is, you can try, or just change steps to avoid it happening.#2020-02-1100:03alexmillerlooking at the clojure script, I think it will still do staleness checking (whether one of the deps.edn files is newer than the cached classpath file), and potentially download and create an unused classpath file even if you -Scp#2020-02-1100:06alexmillerthat seems like a bug, I don't think there's any reason to do that#2020-02-1100:14andy.fingerhutI was thinking one could use clojure -Spath ... > mypath.txt when you had network connections allowed, and "java -cp java command where you were not supposed to make network connections, but sure, making network connections unnecessarily in clojure / clj programs seems like a bad idea.#2020-02-1100:19alexmillerusing clojure -Scp for the latter should have that effect#2020-02-1105:21richiardiandreaMaybe it is completely unrelated, but here we discovered that our Java mvn builds were throwing warnings cause we were still using user> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user> (add-lib 'clj-python/libpython-clj {:mvn/version "1.36"})
user> (require '[libpython-clj.require :refer [require-python]])
...
Error: failed to access class com.sun.jna.CallbackReference#2020-02-1104:24cartesian-theatricsWorked fine after restarting the repl with the dep.#2020-02-1104:24cartesian-theatricsAre there plans to ever merge add-lib into master?#2020-02-1104:28alexmillerit will probably be added in some form, but may change in api/impl#2020-02-1104:30cartesian-theatrics👍#2020-02-1104:30cartesian-theatricsIt's still super useful in it's current form, so thanks for the tool!#2020-02-1104:30alexmilleradd-lib will not always be able to safely add libs as it's trying to resolve new libs in terms of the already loaded library set#2020-02-1104:33alexmillerI'm not sure how the native stuff interacts with any of that either#2020-02-1104:34cartesian-theatricsI see. I'm not well versed on how deps are resolved, but I can imagine there's all sorts of edge cases. Thankfully it does very often work.#2020-02-1104:39seancorfieldAgree. I do all my dev work with Atom/Chlorine + REBL and the add-lib branch of t.d.a. and I love being able to add new dependencies on the fly! Every now and then I add one and get false back (indicating it didn't add it for some reason) but mostly it works exactly as expected.#2020-02-1104:40seancorfieldIt would be nice if, instead of just false, add-lib returned more information about why it didn't add a dependency.#2020-02-1104:41cartesian-theatricsFor sure, at some point think it would be great to give it (the feature) some signal boosting, especially if we can squash more of the edge-case issues. It really helps with flow.#2020-02-1104:41alexmillerwell, it doesn't actually know why#2020-02-1104:43alexmillerit takes all currently loaded libs + the new lib and then does a full resolution, then compares the result with the prior set#2020-02-1104:43cartesian-theatricsIs there a conceivable "brute force" / descructive approach?#2020-02-1104:44seancorfieldFair enough @alexmiller -- it can't report what it does know! 🙂#2020-02-1104:45alexmillerthere are multiple possible reasons why you wouldn't end up with something new#2020-02-1104:46alexmillerbecause everything already loaded is "pinned", adding a different version of something already loaded isn't possible (starting from a real initial dep set doesn't have those constraints, so you can use the version selection to resolve)#2020-02-1104:49cartesian-theatricsI see. Sounds like the current form is about as good as we can do then.#2020-02-1104:50seancorfield@alexmiller Here's a repeatable case of failure, from an empty REPL with the add-lib branch:
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'jedi-time {:git/url "" :sha "8e756ecb71bbfa0b081e00d71a21c47037f1eae4" :deps/manifest :deps})
Cloning:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Checking out: at 8e756ecb71bbfa0b081e00d71a21c47037f1eae4
false#2020-02-1104:51seancorfieldThis works fine as a regular dep:
$ clj -Sdeps '{:deps {jedi-time {:git/url "" :sha "8e756ecb71bbfa0b081e00d71a21c47037f1eae4" :deps/manifest :deps}}}'
Clojure 1.10.1
user=> (require '[jedi-time.core :as jdt])
nil#2020-02-1104:52seancorfieldIs it due to needing the :deps/manifest override?#2020-02-1104:55seancorfield(I see the add-lib branch is about two months behind master at this point)#2020-02-1105:09seancorfield(`add-lib` works if I use {:mvn/version "0.1.4"} instead of the git coordinates)#2020-02-1105:15andy.fingerhutOK, 13K GitHub repositories cloned, with URLs from Clojars projects, of which 12K have project.clj files in their home directories, ~500 deps.edn files in their home directories. More stats later.#2020-02-1105:16seancorfield@andy.fingerhut And the other ~500 are... boot? Or a mixture?#2020-02-1105:18andy.fingerhutDon't know yet. What file name should I look for? Is it boot.edn?#2020-02-1105:18seancorfield(are you counting deps.edn in projects that also have project.clj?)#2020-02-1105:18seancorfieldbuild.boot#2020-02-1105:18andy.fingerhutNo combinations checked for yet, but will do so.#2020-02-1105:18andy.fingerhutThat was just raw count of those files in root dirs.#2020-02-1105:18seancorfieldSweet! You're a rock star!#2020-02-1105:19andy.fingerhut635 build.boot files. Again no stats on projects with different combinations of those. As I said, bigger more detailed report I will send a link later, after I find out how many of these project.clj files can be read with clojure.edn/read#2020-02-1105:20andy.fingerhutAnd then I may want to get fancy and report not only the fraction, but also the weighted fraction, weighted by something like Clojars download counts.#2020-02-1105:20seancorfieldHmm, so CLI/`deps.edn` is already almost as popular as Boot. Interesting.#2020-02-1105:21andy.fingerhutAnd of course many of these repos may not have been updated for years...#2020-02-1105:22seancorfieldGood point. Year-by-year stats based on last update would be interesting if you're thinking of going that way 🙂#2020-02-1105:24andy.fingerhutThat may be interesting enough to spend the extra time to get it.#2020-02-1105:26alexmillerdeps is way more than boot in the 2020 survey#2020-02-1105:27seancorfieldGood to know @alexmiller -- did you see my add-lib issue above (now scrolled off)#2020-02-1105:32alexmillernew deps-mgmt question was lein 84%, deps 49%, boot 6%#2020-02-1105:35seancorfieldWow! That's an impressive uptake!#2020-02-1105:36alexmillerand (obviously) that's multi-select, so presumably a lot of people using both lein and deps#2020-02-1105:28alexmillerI saw it, but not a priority for me right now - please file a question at https://ask.clojure.org#2020-02-1118:34seancorfield@alexmiller I went ahead and filed a JIRA on TDEPS: https://clojure.atlassian.net/browse/TDEPS-149#2020-02-1118:50alexmillerThx#2020-02-1105:29seancorfieldGotcha. I'll see if I can narrow it down a bit first. Should git deps work, in general with add-lib?#2020-02-1105:30alexmilleroff the top of my head, sure?#2020-02-1105:33alexmillerit's using the same resolve-deps as normal deps, so can't think why it wouldn't#2020-02-1105:34seancorfieldThanks.#2020-02-1107:49robert-stuttafordif i had an app project and a library project, and the library project had deps.edn aliases, could i invoke those from the app project?#2020-02-1108:17dominicmNope.#2020-02-1108:34robert-stuttafordthanks for confirming!#2020-02-1109:25robert-stuttafordis there a canonical reference on how to use deps.edn git coordinates with private github repos?#2020-02-1109:27dominicmI think it's just as simple as using the ssh url#2020-02-1109:38robert-stuttafordthanks!#2020-02-1113:19souenzzoHello
$ clj -Srepro -Sdeps '{:paths ["a" "b"] :deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}' -Spath | tr ':' '\n'
a
b
/home/souenzzo/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/home/souenzzo/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/a
/home/souenzzo/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/b
/home/souenzzo/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/home/souenzzo/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
a b paths from "root project" are "chained" to clj-kondo/lsp-server . I think that is a issue, maybe related to TDEPS-52
https://clojure.atlassian.net/browse/TDEPS-52#2020-02-1113:23souenzzoI expect /home/souenzzo/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/src as the default :paths is ["src"]
May also be nice to control :deps/paths 👀#2020-02-1113:35alexmillerwhat version of clj are you using?#2020-02-1113:36alexmillerclj -Sdescribe will tell you#2020-02-1113:37alexmillerif you're not using latest (1.10.1.507) could you update and try again?#2020-02-1114:11souenzzoI'm on 1.10.1.507
I posted a "onliner" so it's easy to rest in many versions#2020-02-1113:38alexmillerTDEPS-52 is long fixed, so I'm wondering if you're on an older version#2020-02-1113:52borkdudethis is my output with the newest (507):
$ clj -Srepro -Sdeps '{:paths ["a" "b"] :deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}' -Spath | tr ':' '\n'
Cloning:
Checking out: at 4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5
a
b
/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/a
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/b
/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar#2020-02-1113:53borkdudeso it might be an issue with :paths + :deps/root ?#2020-02-1113:55alexmillerdunno, must be a bug, please file either a jira or a question on https://ask.clojure.org#2020-02-1113:55alexmilleror I guess I'll just make a jira, nvm#2020-02-1113:57alexmillerhttps://clojure.atlassian.net/browse/TDEPS-148#2020-02-1113:57alexmillerhttps://clojure.atlassian.net/browse/TDEPS-148#2020-02-1114:15souenzzoatlassian login system is a pure mess
I have 3 accounts (clojure, work1, work2)
When I'm authed as ẃork1 and try to login to vote on clojure, it blows a auth error
then I need to go to work1 workspace, logout, login in clojure workspace, then(.................) 😓#2020-02-1114:22souenzzoafter create a new browser to login in jira, there is no vote option to vote on TDEPS issues?#2020-02-1114:28alexmillerthe jira auth is for sure, not good, particularly cross accounts. identity is tied strongly to email but the gsuite tie-ins make everything pretty bad. I don't have a good solution to that other than private browser window#2020-02-1114:28alexmillerif you have a clojure jira logon, you should be able to vote. that said, they have been actively destroying the user experience with their "new ui" issue view. it could just be that it's buried a menu level or two down#2020-02-1114:29alexmillerI'm using the old ui, so not sure I can point to how to do it#2020-02-1114:38borkdudeI have an old account, but I'm not able to vote for a long time anymore, at least, can't find that option#2020-02-1114:44souenzzookay after spawn 5 firefox isolated instances and login with all my credentials, I'm sure that voting is disabled.
My account is {:tag :mailto:souenzzohotmail.comsouenzzohotmail.com, :attrs nil, :content nil} with "raw password" (without microsoft auth).#2020-02-1115:17alexmillerthere is no way in jira to disable voting for a user#2020-02-1115:17alexmillervoting is enabled in our jira product instance#2020-02-1115:18alexmillerwhether you can vote is dependent on whether you can edit issues and I have it configured in all projects that if you are a logged in user, you can edit issues#2020-02-1115:18alexmilleryour acct looks normal - you have jira product access and good default roles like other users#2020-02-1115:19alexmillerif you go to something like https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-132 - you don't see a link "Vote for this issue" by the votes field on the right?#2020-02-1115:20borkdudeif voting is so hard to find, maybe we can just regard it as non-existing#2020-02-1115:22alexmillerit's a link at the top with the words "Vote for this issue" which seems pretty easy to find to me#2020-02-1115:23alexmiller(note that you can't vote for issues you create)#2020-02-1115:23borkdudeI have tried to find it multiple times, and apparently I'm not the only one. I gave up on it.#2020-02-1115:24alexmillerwell there must be something I'm missing then#2020-02-1115:24alexmillerthere is no intent for it to not be there#2020-02-1115:25borkdude#2020-02-1115:26andy.fingerhutI see how to vote using my JIRA account when looking at a ticket using the old view, but not the new one#2020-02-1115:27andy.fingerhutThere is a link "See the old view" near the top center of the frame.#2020-02-1115:27alexmilleryeah#2020-02-1115:27alexmillerif you go to https://clojure.atlassian.net/secure/ViewPersonalSettings.jspa you can configure this#2020-02-1115:28alexmillerturn OFF "New JIRA Issue view" - when you do, it asks you why. tell them you can't vote. :)#2020-02-1115:28andy.fingerhutBut agreed without explicit instructions somewhere, most people won't bother hard trying to find it.#2020-02-1115:31alexmillerhere's a jira issue in jira about this https://jira.atlassian.com/browse/JRACLOUD-71251#2020-02-1115:31alexmillersorry, I turned this off so long ago I forgot about it#2020-02-1115:31alexmiller278 votes to enable voting#2020-02-1115:32borkdudeok, I reverted to the old view of which I was unaware it existed and now I see the votes again#2020-02-1115:32andy.fingerhutThose were some determined voters, who figured out how to vote on it 🙂#2020-02-1115:32borkdudelol#2020-02-1115:35alexmilleralso, I as an admin have no ability to change the default - only users can change their own setting. there is another jira about that.#2020-02-1115:35borkdudemaybe we can migrate to clubhouse, I've heard they built it in clojure, so it must be good#2020-02-1115:38alexmillerI'm done migrating issue data for a while :)#2020-02-1115:38borkdudejust start over. if it's important, the issues will pop up again 😛#2020-02-1115:38alexmillerit was nice that they added the new issue, with fewer features, and made it the default, and prevented admins from changing that default#2020-02-1115:39alexmillermany years ago, I had a boss that literally deleted the issue database with that theory#2020-02-1115:39borkdudehow did it play out?#2020-02-1115:40alexmillernot great#2020-02-1115:40borkdudeif users have a stake in the issue that was posted, you could also trust on them to migrate the issue, if they found it important#2020-02-1115:40borkdude(I'm speaking as devil's advocate)#2020-02-1115:40alexmillermostly, they just felt like we didn't care about their problems#2020-02-1115:41alexmillerand we had no way to notify them when we did happen to fix stuff they'd logged earlier#2020-02-1115:41borkdudehmm, right, if it's customer issues, it might be different#2020-02-1115:42alexmillerthere is some happy medium where you mark things as "won't fix" more aggressively than feels comfortable I think#2020-02-1114:01borkdude@souenzzo As a workaround, this seems to work:
$ clj -A:lsp:dev -Srepro -Sdeps '{:aliases {:dev {:extra-paths ["a" "b"]} :lsp {:extra-deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}}}' -Spath | tr ':' '\n'
a
b
src
/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/src
/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar#2020-02-1114:04borkdudeThis one also works:
$ clj -A:dev -Srepro -Sdeps '{:aliases {:dev {:extra-paths ["a" "b"]}} :deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}' -Spath | tr ':' '\n'
a
b
src
/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/src
/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar#2020-02-1120:13hlolli(at)alexmiller, as a continuation of the bugs I mentioned a bit last night related to nixpkgs. It seems as if this isn't related to lumo, but clojure-tools itself. Since I've been tagged a lot on various tickets on nixpkgs due to lumo, I#2020-02-1120:13hlolliI'd love to find a workaround. (sorry pressed enter too early)
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/clojure/default.nix#2020-02-1120:14ghadiIf I understand this correctly, it's a name resolution issue#2020-02-1120:17hlolliso the problem is that the current clojure, and newer ones for that matter, will by default call maven when called. While a clojure-tools does provide all the classpath needed to run the clojure executeable. So it's basically bootstrapped as it is released. The way to workaround this, would be to force clojure to not check for updates, and just use the bootstrapped jar on runtime for its own environment, and if the user then later wants to resolve dependencies with another deps.edn, it will in those cases ofc continue it's normal behaviour. The issue here is only the runtime environment of clojure and not for another project.#2020-02-1120:18andy.fingerhutIf your deployed application uses a command like java -classpath your:class:path:here and whatever other options you want, then it should not call Maven stuff, unless your application does.#2020-02-1120:19andy.fingerhutAlex noticed last night that clojure -Scp your:class:path:here might still call Maven and make network connections for that, which he considers a bug, I think.#2020-02-1120:19hlollito the default.nix I posted above, I added
doInstallCheck = true;
installCheckPhase = ''
CLJ_CONFIG=$out CLJ_CACHE=$out/libexec $out/bin/clojure -Spath
'';
this fails the same way, maven not found.#2020-02-1120:20hlolliit can't run itself without internet it seems#2020-02-1120:20andy.fingerhutHave you tried java command instead?#2020-02-1120:20andy.fingerhutmight require changes to other command line options, too, so not necessarily a one-line change.#2020-02-1120:20hlollinot sure it matters because clojure is just a shell for calling java anyway#2020-02-1120:20hlolli#! /nix/store/b85h3rw3y20hyj6s2aqgvlm26a8v61gy-bash-4.4-p23/bin/bash -e
export PATH='/nix/store/rz4aba9s8895b69l3kmx8qb6ir8di2xv-clojure-1.10.1.507/bin:/nix/store/6xxk9hynq5q5nky6wh8l7k4y95y3aksv-rlwrap-0.43/bin:/nix/store/kqr55dy5zjmhklzkkwc85ghf7yxk11qq-openjdk-11.0.6-ga/bin'${PATH:+':'}$PATH
exec -a "$0" "/nix/store/rz4aba9s8895b69l3kmx8qb6ir8di2xv-clojure-1.10.1.507/bin/.clojure-wrapped" "#2020-02-1120:20andy.fingerhutdid you read my message above, completely?#2020-02-1120:21andy.fingerhutthe one that mentioned Alex by name#2020-02-1120:21hlolliahh yes ^#2020-02-1120:22ghadidoes the nixpkg build system intentionally prevent network calls?#2020-02-1120:22ghadi(I know very little about nix)#2020-02-1120:22hlolliyes, it allowes network calls, you just need to specify the endpoints by hand. This is by design ("purely functional" buzzwording)#2020-02-1120:23ghadimakes sense#2020-02-1120:24andy.fingerhutSo it seems you can wait for a bug fix to clojure command, if one is forthcoming, or try java command.#2020-02-1120:24hlollijust thinking outside the box until a fix arrives, perhaps making some sort of .cpcache file to prevent clojure from attempting to call maven#2020-02-1120:25andy.fingerhutSure, if you want to run experiments like that, go for it. If you do not know clojure internals, you are doing black box testing.#2020-02-1120:26hlollihence me asking, I'm also ok with solving this on a slow tempo, I'd just remove lumo temporarily in that case#2020-02-1120:27andy.fingerhutYou could also add https://repo1.maven.org to explicit whitelist for nixpkg, if you have such access, and then rinse and repeat as other unexpected host accesses arose.#2020-02-1120:29hlollithat I can't do, but what I can do, is getting all the jars clojure is attempting to download from nix and put them anywhere visible to clojure. But I just worry if adding -Spath to the clojure wrapper will interfere with normal useage (like two -Spath would be then plausable input parameter to clojure if user provides one and nix one)#2020-02-1120:30hlollithat is an alternative, but then again, all the files clojure needs to run are already downloaded, so I was hoping to do --offline-mode or something similar until going into more unknown terretory#2020-02-1120:31andy.fingerhutIt might work to run an experiment where you run a command like ps axguwww on the deploy box while your failing clojure command is running to see the full command line and options of all processes, and see what command line options are actually being used.#2020-02-1120:32hlollinice, if I find the root of the bug, there's always a possibility of patching (or using forked repo etc..). I'll try this! thanks!#2020-02-1120:32andy.fingerhutI don't know your time tolerance level for experiments 🙂#2020-02-1120:33hlolliI'll write a react app in assembler for fun 😄#2020-02-1120:33ghadihaha#2020-02-1120:38alexmillerthe fix in clojure is just extending the flag checks around the tool invocation#2020-02-1120:38alexmiller(and the equivalent in the windows script)#2020-02-1120:38alexmillerif someone wants to make a patch, I will likely be doing a release this week and would be happy to include#2020-02-1120:44alexmillerhttps://clojure.atlassian.net/browse/TDEPS-150#2020-02-1120:48hlolliperfect, I think this describes it perfectly#2020-02-1121:35hlollijust reading trough the shell script quickly, everything makes total sense, and I like how the checksum is based on arguments and environment. Because it's hard to know if the current environment is "stale", so I'd be reluctant to want to make it less strict. Just a simple
if [[ "$stale" = true && "$describe" = false ]]; then
to
if [[ "$stale" = true && "$describe" = false && "$offline" == false ]]; then
or "$dont_recalculate_classpath" instead of $offline#2020-02-1121:36hlolli> > ...would be enough for my usecase#2020-02-1121:46alexmillerdon't worry about the staleness check, just add it as a flag that affects the running of the tool#2020-02-1121:47alexmillerthis line: if [[ "$stale" = true && "$describe" = false ]]; then#2020-02-1121:47alexmillerand not -Scp ...#2020-02-1121:48alexmillerwhatever the bash is for that#2020-02-1121:50hlolliI see, are there any use-cases of people partially wanting to add classpath but otherwise rely on cached (or yet to be cached) classpath?#2020-02-1121:51hlollithere's extra-classpaths for that I guess, so -Scp would be total opt-out of make-classpaths mechanism#2020-02-1121:52alexmilleryes, that's it's purpose imo#2020-02-1121:53alexmillerso should shut off making a cp#2020-02-1121:53hlollimakes sense to me, with extra-classpaths well documented, shouldn't come as a surprise#2020-02-1122:31hlollibug fixed https://github.com/NixOS/nixpkgs/pull/79868 easy 😛#2020-02-1122:38alexmillerare you going to make a patch?#2020-02-1122:39hlolliyes! be happy to#2020-02-1122:46andy.fingerhutWow, just the raw numbers of issues, PRs, and commits on that project are mind-boggling. Sounds like a buzzing hive of activity: https://github.com/NixOS/nixpkgs#2020-02-1122:55alexmillerlooks just like the homebrew core tap :)#2020-02-1121:48andy.fingerhutWhen investigating which project.clj files are readable via clojure.edn/read, I have noticed a couple of projects that have both deps.edn and project.clj, and their project.clj is not EDN because it has non-EDN executable Clojure that reads the deps.edn file. I am guessing that if there is both a project.clj and deps.edn file in the root directory of a project's source code, then it doesn't matter whether project.clj is EDN or not?#2020-02-1121:51andy.fingerhutI've also noticed @U04V15CAJ that you have joined "`#=` club" for several of your project.clj files. I have seen plenty that have an eval-able Clojure expression in place of a version string, that evaluates to a version string, that do not prefix that expression with #= , but simply have the expression. I haven't verified myself whether that actually works, but might be worth testing to see if it does. Either that or plenty of projects left their project.clj files broken in that way, or it only worked in some versions of Leiningen, etc.#2020-02-1122:05borkdude@andy.fingerhut I have joined that club for some projects but for e.g. this one I changed that to templating later: https://github.com/borkdude/clj-kondo.lsp/blob/master/script/update-project.clj
which is I think the better thing to do. I haven't heard any problems about the read-eval approach tooling-wise though#2020-02-1122:05borkdudeI think I only used the read-eval because I didn't know any better#2020-02-1122:15andy.fingerhutThere is no problem tooling-wise that I am aware of. This is just me noticing things while checking out ~12,000 project.clj files in projects, to see which are clojure.edn/read'able, which not, and an a few cases they are not, looking at why. Alex Miller mentioned the possibility of having deps.edn use clojure.edn/read for project.clj files if a project had no deps.edn, and I'm still doing the data collection....#2020-02-1123:03seancorfieldI've never seen #= in a project.clj but I've seen plenty of ~ to evaluate inline expressions... which wouldn't read as EDN, right?#2020-02-1121:53alexmillerdeps.edn is preferred#2020-02-1121:53alexmillerso if it has that, project.clj won't be used#2020-02-1122:23andy.fingerhutThanks for confirmation. Will use that in my summary of findings, and mention it.#2020-02-1123:07seancorfieldSome projects will have both even if project.clj isn't using machinery that reads deps.edn since some people add the latter in response to folks complaining that their projects can't be used by the CLI as git deps 🙂 So they'll start off compatible in terms of general dependencies but may drift apart later if the maintainer forgets to update deps.edn when they update project.clj (in the same way that maintainers of Contrib projects added project.clj files alongside pom.xml files and then the POM was updated but not the project.clj file!). So it's worth mentioning that they could differ but, hopefully, such projects would be a tiny proportion of the whole...#2020-02-1123:14andy.fingerhutUnderstood they can differ, in the absence of such automated methods. Right now I'm partly documenting/understanding, and also wanting to make the results as accurate as I know how, by counting a project with both deps.edn and project.clj as "good for tools.deps to read project.clj file, in the last resort"#2020-02-1123:18andy.fingerhutSo I am planning to soon calculate which subset of these projects have each of the 16 possible combinations of files in their repository home directory: project.clj deps.edn build.boot pom.xml. Any others of interest to throw into that mix?#2020-02-1123:18andy.fingerhutThen I plan to report stats for, within each of those 2^4=16 groups, which ones have EDN-readable project.clj files.#2020-02-1123:29seancorfieldThose are the only four files I can think of that are interesting. It'll be interesting to see which notionally Clojure projects have none of those four (Gradle-based, maybe?).#2020-02-1200:21alexmillerI'm really concerned most about project that have only a project.clj (and no deps.edn or pom.xml)#2020-02-1200:24penryuFirst, wanted to thank @seancorfield for his dot-clojure. 😄
Also wondering if tools.deps has anything similar to leiningen's :injections functionality?#2020-02-1200:25alexmillerno#2020-02-1200:26alexmillerinstead of running a repl, you could launch your own program that started its own repl that did something similar#2020-02-1200:27penryuack. are there any workarounds? something as simple as (require '[clojure.pprint :refer [pp]]) could save a lot of typing over time.#2020-02-1200:28penryuTrue. I'm in the process of converting from emacs/cider to vim/conjure. I'll pursue that route.#2020-02-1200:28penryuThank you!#2020-02-1200:29alexmillerlike clojure.main effectively just does this:
(repl :init (fn []
(initialize args inits)
(apply require repl-requires)))
#2020-02-1200:30alexmillerthe initialize part will already be handled by the main wrapper, so don't need that line#2020-02-1200:31alexmilleryou'd want to supply the :eval hook: #(do (injections) (eval %))#2020-02-1200:32alexmillersomething like that, the default is just eval#2020-02-1200:33penryu... I'm afraid that's a little beyond my level of knowledge of the repl/clojure.main at the moment, but I'll take some time this week to dig in to how that would work in practice.#2020-02-1200:33alexmillerthat's all the code#2020-02-1200:33penryuAppreciated, nonetheless!#2020-02-1200:35penryuthe eval hook is defined in deps.edn?#2020-02-1200:40alexmilleras a command line:#2020-02-1200:40alexmillerclj -e "(require '[clojure.main :as m]) (m/repl :init #(apply require m/repl-requires) :eval (fn [form] (do (require '[clojure.pprint :refer [pp]]) (eval form))))"
#2020-02-1200:40alexmillerwhich you can then fold into an alias if you like#2020-02-1200:43alexmiller;; deps.edn:
{:aliases {:repl {:main-opts ["-e" "(require,'[clojure.main,:as,m])(m/repl,:init,#(apply,require,m/repl-requires),:eval,(fn,[form],(do,(require,'[clojure.pprint,:refer,[pp]])(eval,form))))"]}}}#2020-02-1200:44alexmillerthanks to the Corfield Comma for serving as bash-safe whitespace... :)#2020-02-1200:43alexmillerthen clj -A:repl#2020-02-1200:44alexmilleryou could fold that tiny program into a tool that abstracted out the injections rather than hard-coding them and clean this up a bit, but that's the core of it#2020-02-1201:07seancorfieldWait... pp is available directly in the REPL started by clj already...#2020-02-1201:08seancorfield> clj
Clojure 1.10.1
user=> *clojure-version*
{:major 1, :minor 10, :incremental 1, :qualifier nil}
user=> (pp)
{:major 1, :minor 10, :incremental 1, :qualifier nil}
nil
user=>
#2020-02-1201:09seancorfield@penryu What injections do you want in the Clojure REPL, specifically? (given that pp already seems to be referred in by default)#2020-02-1201:14alexmillerit's only available in user#2020-02-1201:14alexmillerinjections makes it available in every ns#2020-02-1201:15alexmillerI dropped the above in a blog in case it's helpful for posterity https://insideclojure.org/2020/02/11/custom-repl/#2020-02-1201:20kennyfyi#2020-02-1201:44alexmilleryeah, I know :)#2020-02-1201:15alexmillerand so I can just point someone to it next time :)#2020-02-1201:17penryuThanks for the explanation, @alexmiller!#2020-02-1201:18penryu@seancorfield might also be something like tools.logging or spyscope (as recommended in a post), though I’m having problems getting either of them to work right now.#2020-02-1201:20seancorfield(I never used :injections in all the years I used Leiningen and I'm not sure not sure I'd want a bunch of symbols auto-inserted into every namespace while I'm working so...)#2020-02-1201:44alexmillerwell, me neither :)#2020-02-1201:58penryuI just found out about :injections today, but I'm getting a feel for the [mis]feature parity of deps vs lein.#2020-02-1202:02alexmillerthe philosophy of clj is much more to give you the tools to solve your own problems, not to exhaustively provide every possible feature#2020-02-1203:14penryuThanks again for your response. I know leiningen has accumulated a lot of feature-cruft, and I appreciate how clj factors that out. Especially, thank you for the workaround in your blog post!#2020-02-1201:26penryuYes, it would need to be sparse in the extreme. But certain things (like pp or #spy) might be reasonable. And I intended to bury it behind an alias (`-A:spy:rebel` or just fold it into rebel) anyway, never in a clean clj session.#2020-02-1202:05ghadiWhat is the reason the Corfield comma is necessary in the first place?#2020-02-1202:24seancorfieldRound-tripping to text files and back into bash (and then into Clojure as command-line arguments).#2020-02-1202:24seancorfield(shell escaping is... facepalm )#2020-02-1202:25ghadiis there a technical reason it can not be done with full fidelity, or has no one dived into the GNU Bash Info page?#2020-02-1202:27ghadior whatever Windows uses for bash 😂#2020-02-1202:37seancorfield@ghadi The "official" clj-on-Windows uses Powershell -- which has different quoting rules from bash (you end up triple-quoting some things -- ugh!)#2020-02-1202:27ghadiwhich I think is now... bash#2020-02-1202:27andy.fingerhutI suspect it could be done with full fidelity, but the prevalence of multiple different versions of bash on Mac OS versus different flavors of Linux adds a level of difficulty to the problem. Also not just spaces, but single and double quotes, dollar signs, braces, ... It can make a grown person weep.#2020-02-1202:28andy.fingerhutNot sure it is a hill I want to even try to climb, nor die on.#2020-02-1202:29seancorfield@ghadi I'm sure Alex would welcome patches for it 🙂#2020-02-1202:47alexmillerThere are patches for it. They are all scary #2020-02-1202:30seancorfield"It's hard!" is the technical reason I've heard from several people.#2020-02-1202:30andy.fingerhutI suspect it would be difficult to maintain the shell scripts, too, and lead to copious notes on what works and what doesn't across the platforms.#2020-02-1202:30ghadiI'm thinking of tools that accept main arguments (like Docker) usually accept a canonical form: an array of strings (which is sort of punting)#2020-02-1202:30seancorfieldThe complexity comes from clojure storing all that stuff in text files in .cpcache and then reading it back on subsequent runs.#2020-02-1202:31andy.fingerhutI know reasonable people might disagree with this statement, but bash is just barely a programming language.#2020-02-1202:33andy.fingerhutI will end my rant-ish statements on that topic for the evening 🙂#2020-02-1202:38andy.fingerhutI lied. As a starting point for consideration, bash --version on the penultimate macOS version (Mojave 10.14) gives this output:
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
Copyright (C) 2007 Free Software Foundation, Inc.
#2020-02-1202:39andy.fingerhutOn Ubuntu 18.04 Linux:
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
#2020-02-1202:39andy.fingerhutThere's 9 years of variation possible right there.#2020-02-1218:44slipsetAlso, Catalina wants you over on zsh because the more modern bash have a license that doesn’t please Apple#2020-02-1202:49alexmillerI actually have a recently raised alternative to the spaces issue that avoids all the escaping that I’m exploring. #2020-02-1202:50alexmillerRaised by rich and Stu for other purposes but happens to also cover this#2020-02-1206:29Nate Suttonquoting things for bash is really not all that hard, in my experience, so long as you don't want any shell evaluation. you wrap each thing in ' and then each single quote within it should be replaced with '\''#2020-02-1206:30Nate Suttondouble quoting introduces a billion corner cases, though#2020-02-1206:34Nate Suttonhttps://wiki.bash-hackers.org/syntax/quoting#strong_quoting#2020-02-1206:35Nate SuttonI don't really understand how round tripping affects things#2020-02-1206:35seancorfield@nate_clojurians Then you should write up a patch for clojure and/or tools.deps.alpha that fixes these problems 🙂#2020-02-1207:37sogaiucuz this is ridiculous: https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#escaping-quotes#2020-02-1207:38sogaiubut actually, may be that's a diff issue?#2020-02-1207:40andy.fingerhutThat shows differences in command line parsing for different command shells on Windows.#2020-02-1207:41andy.fingerhutThere are differences between command line parsing for different command shells on Linux/Unix/macOS, too, but bash is a pretty commonly used one, and what clj/clojure scripts are currently written in.#2020-02-1207:42andy.fingerhutThere are things involving single quotes, double quotes, white space, etc. that can be tricky to get correct even in bash, without considering the flavors of Windows command shells described at the link you gave.#2020-02-1207:44sogaiui guess if there ever is a cross-platform shell that gets adopted may be this issue will be less of a problem#2020-02-1207:45andy.fingerhutbash exists on Windows in fairly easily obtainable form. That makes things more common with other platforms, if you use bash on Windows, but it doesn't eliminate the existing difficulties with bash#2020-02-1207:47sogaiui use bash on windows via the git for windows installation -- but i hesitate to use it as a shell all of the time.
i think there's a chance that powershell core might at some point be a nicer alternative.#2020-02-1208:02andy.fingerhutOn Linux as well? And macOS? That seems unlikely to me, but stranger things have happened.#2020-02-1208:04sogaiufirst paragraph here seems to suggest some level of support for the 3: https://github.com/PowerShell/PowerShell#-powershell
i've not tried it on macos, but have on linux#2020-02-1208:06sogaiui'm not a power bash user but have dealt on and off with shell scripts since the 90s, and though i've historically not been an ms fan, the powershell stuff (at least in core) seemed way more sane to me (though i've used it far less than bash, sh, zsh, etc.)#2020-02-1208:07sogaiuit's easier to do so when you start over ofc#2020-02-1208:50penryuAt what point is it worth reading code from a file instead of a command line argument? It seems tools.deps has drawn that line in the sand.#2020-02-1208:55andy.fingerhutI am not sure what you mean by the line in the send. The clojure command lets you give Clojure expressions on the command line after the --eval option (in the init-opts part of the command line args), and/or a file name to evaluate code from named path in the help output by clojure -h#2020-02-1208:56andy.fingerhutBy the line in the sand, do you mean it lets you pick what you want?#2020-02-1218:09seancorfieldCaching. clojure does two passes -- and invokes the JVM twice -- if the classpath/main/JVM options are not cached (in a file). It does just one pass with the cache in place. That's a big deal.#2020-02-1214:46sparkofreasonExpanded version of my earlier question about javac: suppose I want to get in between the resolve-deps and make-classpath steps before running clj, say to do some other processing on files fetched from git, add resulting artifacts to the classpath. Are there any examples of tools (or whatever) doing such?#2020-02-1215:11delaguardoYou can try https://github.com/EwenG/badigeon
I’m not sure this is exact solution for your specific task, but could provide some missing tool’s parts#2020-02-1215:13sparkofreasonYes, assuming nothing else existed, that was going to be my starting point. Very useful stuff.#2020-02-1214:55dominicmNot that I'm aware of, that's a new, separate tool.#2020-02-1214:55dominicmHowever, I think that would be really useful for the javac case.#2020-02-1214:56dominicmOther tools are generally shell scripts which you run first, but that reduces the smooth integration with cider, etc.#2020-02-1218:40andy.fingerhutOK, I do not have stats weighted by number of http://Clojars.org project downloads right now, but I do have some stats showing that out of 11,736 Github projects I successfully cloned, and that is the subset of them that have a project.clj file but no deps.edn file in their root directory, 91.8% of those project.clj files were readable using clojure.edn/read , and had their defproject form first, and a reasonable-looking value for the :dependencies key (or no dependencies key, which is completely correct and fairly common). More data and steps I used to collect it here: https://github.com/jafingerhut/haironfire#2020-02-1218:42alexmillercool (and nice repo name :)#2020-02-1218:42andy.fingerhutThe most common reason that a project.clj file threw an exception while attempting to use clojure.edn/read to read its contents: regex literals are not valid EDN. Those are reasonably common in project.clj files, for things like jar inclusions/exclusions, or other kinds of file name selection sets.#2020-02-1218:43alexmillerthat's kind of interesting#2020-02-1218:44alexmillerI wonder if you could hack the edn reader to read those into tagged literals or something#2020-02-1218:44alexmillerhow much additional % does that cover?#2020-02-1218:46andy.fingerhutGood question, which I thought of attempting, but haven't yet. That was the first thing in the project.clj file that caused clojure.edn/read to throw, and there could be others after that, which would only be straightforward to calculate with such a modified version of clojure.edn/read#2020-02-1218:47alexmillersuch a thing could possibly even make its way into clojure's real edn reader#2020-02-1218:49ghadiregexes aren't portable though#2020-02-1218:49alexmillerI didn't say make it into edn#2020-02-1218:50alexmillerthere are lots of things the clojure edn reader does beyond the edn spec#2020-02-1218:50ghadihmmm.jpg#2020-02-1218:50alexmillerand I'm not suggesting reading it into java pattern, but into a tagged literal#2020-02-1218:50ghadisame thing for the var dispatch I was talking to you about#2020-02-1218:50alexmiller#regex or whatever#2020-02-1218:51alexmillerless sure on vars if that makes sense as you're then connecting to code#2020-02-1218:53ghadi(tangent, but I want to have the choice of whether to connect to code or read as tagged)#2020-02-1219:00dominicmEdamame can read regex I think#2020-02-1219:04andy.fingerhutBased on the numbers I have seen, if the project.clj files that threw exception due to regex are edn-readable in all other ways except the regex literals, I think the most the numbers above could improve would be from current 91.8% up to close to 95% or 96%, but that would be noticeable, I think.#2020-02-1219:06andy.fingerhutSounds like a straightforward thing to hack up locally on my system. I will let you know if I try it out. If anyone hacks up a version of clojure.edn/read before I do, let me know.#2020-02-1219:08ghadihttps://github.com/clojure/tools.reader/blob/master/src/main/clojure/clojure/tools/reader/edn.clj#L345-L354#2020-02-1219:08ghadi@andy.fingerhut you could switch to tools.reader and monkey patch that to read regex + vars#2020-02-1219:09andy.fingerhutCool, that may be the quickest way.#2020-02-1219:09ghadinaughty, but straightforward#2020-02-1219:09andy.fingerhutMaybe I should take the opportunity to see if clojure.edn/read and tools.reader edn reader behave differently on any of these 11,736 files 🙂#2020-02-1221:04sogaiuon a tangential note perhaps, regarding regexes -- there's the regal thing that expresses regexes using plain clojure data: https://github.com/lambdaisland/regal#2020-02-1221:06alexmillerthis is about existing data in project.clj files, so I don't think that helps anything#2020-02-1221:07sogaiuyou're talking about converting regexes as well as whether the result is valid edn, no?#2020-02-1221:08sogaiuanyway, for future cases, it could be useful for folks to express their regex using regal and having a file that is more likely to be edn?#2020-02-1221:23andy.fingerhutThis is experimental exploration here, wondering how we might use a secure safe reader, either existing, or perhaps slightly tweaked, to read existing project.clj files.#2020-02-1221:24andy.fingerhutI doubt we could ever convince hundreds of people to update the project.clj files to use a new regex representation, for something that already works for them today.#2020-02-1221:57sogaiusure -- just musing about future possibilities. sorry for the noise.#2020-02-1222:03andy.fingerhutNo worries. Just hoping the context makes it clear why we're talking like we do 🙂#2020-02-1222:43sogaiuthanks for your ongoing patience and explanatory efforts 🙂#2020-02-1221:47andy.fingerhutFraction of the 11,736 project.clj files that become 'good' in my experiments (readable using clojure.edn/read without throwing exceptions, defproject form is first, value of dependencies key looks reasonable) goes up from 91.8% to 95.7% with a hacked up version of EdnReader class that can read regex literals. Details here: https://github.com/jafingerhut/haironfire/blob/master/README-extended-edn-reader.md#2020-02-1221:52andy.fingerhutAs mentioned in #clojure-dev channel discussion, my 'good' evaluation here currently has no checking for whether the dependencies list is actually the one used by the project, e.g. no checking for or evaluating the effects of Leiningen plugins, e.g. Lein-modules. There are also approximately another 1,000 git repos associated with Clojars.arg artifacts that have no project.clj nor deps.edn files in their home directory. Most of those probably have some kind of hierarchical structure to their files, with one or more independent Leiningen projects in the same source repo. I haven't tried analyzing those.#2020-02-1222:10Nate SuttonI wonder if you could use sci to safely figure out what exactly is not valid edn?#2020-02-1222:13Nate SuttonI mean, beyond the regex literals#2020-02-1222:13Nate SuttonI mean, beyond the regex literals#2020-02-1222:18andy.fingerhutThe results I linked to show number of files that give exception messages of various kinds. The next most common non-EDN things, or at least the ones that occur first in the project.clj files, are mentioned explicitly in my commentary text here: https://github.com/jafingerhut/haironfire/blob/master/README-extended-edn-reader.md#2020-02-1222:18andy.fingerhutShort answer appears to be "in-line eval'able code inside the defproject form"#2020-02-1222:19Nate Suttonahhhhhh#2020-02-1222:19Nate Suttoncool#2020-02-1222:19andy.fingerhutOne could pretty easily extend an EDN-style reader to read those things, too, but I suspect the return-on-time-invested is getting fairly low.#2020-02-1222:26Nate Suttonyeah for sure, definitely diminishing returns at this point, from the looks of it#2020-02-1222:26Nate SuttonI just saw the need for a safe, custom edn reader and wanted to share 🙂#2020-02-1222:27Nate Suttonbut I guess that's actually edamame https://github.com/borkdude/edamame#2020-02-1222:56borkdude> One could pretty easily extend an EDN-style reader to read those things, too, but I suspect the return-on-time-invested is getting fairly low.
@andy.fingerhut That's exactly what edamame already is.#2020-02-1223:00borkdudee.g.:
(parse-string "#=(+ 1 2 3)" {:read-eval true})
;;=> (read-eval (+ 1 2 3))#2020-02-1223:00borkdudebut :read-eval can also be a function that transforms the value in any way you want#2020-02-1223:01andy.fingerhutUnderstood. Part of the context of this discussion, I believe, is what dependencies outside of itself and Clojure that Alex is willing to take on for tools.deps.#2020-02-1223:02andy.fingerhutI do not know his detailed thoughts on the matter, other than some nuanced approximation of "as few as possible".#2020-02-1223:03borkdudeYeah, I understand. I guess you could also have a "forgiving" EDN parser that just skips over what it cannot read. If that leaves a valid :dependencies vector, that's probably all that counts.#2020-02-1223:05andy.fingerhutI suspect the harder part of getting a valid dependencies sequence (one thing I learned is Leiningen accepts, and a few project.clj files actually use, lists or even a map for the value of :dependencies ), is the existence of git repos with a project.clj file in the root directory, and then several more in subdirectories, with interactions between their contents, e.g. via Lein-modules plugin or other mechanisms.#2020-02-1223:05borkdudeso what is the summary of your research, what's the percentage of EDN-readable project.cljs?#2020-02-1223:06andy.fingerhut91.8% of 11,736 project.clj files that are in the root directory of git repos, and those were selected because they do not also have a deps.edn file in the root dir, are clojure.edn/read readable in Clojure 1.10.1. If you extended clojure.edn/read to not throw an exception on encountering a regex literal, it goes up to 95.7%#2020-02-1223:08andy.fingerhutBut as I mentioned two comments ago, the percentage that are EDN-readable isn't all that one cares about if one wants to get correct dependencies from a Leiningen source repo.#2020-02-1223:08borkdudevery cool. I assume you communicated this summary to Alex as well?#2020-02-1223:08andy.fingerhutEarlier today in this same channel.#2020-02-1223:08borkdudetrue, but maybe those multi-project.clj repos are also in the <10% range?#2020-02-1223:08andy.fingerhutI haven't tried to determine that yet.#2020-02-1223:09borkdudeI haven't had time to read slack today. Nice progress!#2020-02-1223:10andy.fingerhutSure, it's been fun hacking away. Also makes me realize that the actual question is harder to answer 🙂#2020-02-1305:45rbergerDoes seancorfield/clj-new have a way to specify copying a binary file from the template source dir to the target dir? The doc kind of says that the render function without a data argument would do that, but it looks like it treats the file as text with linefeeds and corrupts the binary data. Is there a way to do this that still fits into the ->files body?#2020-02-1317:32seancorfield@U07GPFFAS That sounds like a nice enhancement! Feel free to open a GitHub issue. Do you know if either lein-new or boot-new support such a thing? (`boot-new` was originally based on lein-new and clj-new was forked from boot-new but I haven't tracked their progress much since).#2020-02-1318:01rbergerI haven't used the others {lein,boot}-new I'll open a github issue. I guess in the short term I could create another renderer for binary files that would do the copy#2020-02-1318:03seancorfieldFeel free to provide a PR with that enhancement too.#2020-02-1318:08seancorfield@U07GPFFAS Just looked in the code and there's a raw-resourcer which I suspect can be used to copy binary files. I can't say I remember trying that but it's worth a shot.#2020-02-1318:09rbergerIt wasn't clear to me when I looked at that, but will look at it today. Thanks!#2020-02-1318:14seancorfieldhttps://github.com/search?q=raw-resourcer&type=Code -- looks like quite a few Leiningen templates use that so that should point you in the right direction#2020-02-1321:21rbergerLooks like the raw-resourcer does the trick! No need for any changes in clj-new. Just a couple of lines in my template code (https://github.com/omnyway-labs/chromex-shadow-template)
By the way for some reason https://cljdoc.org/d/seancorfield/clj-new is having problems building the docs for the most recent couple of releases of clj-new.#2020-02-1321:25seancorfieldThanks for the heads up. Weird error Failed to read artifact descriptor for com.fasterxml.jackson.core:jackson-databind:jar:2.7.5#2020-02-1321:26seancorfieldHmm, it's leftover in the pom.xml by accident 😞#2020-02-1321:41seancorfieldI've pushed 0.9.0 with that fix but the doc system build process is broken right now (by changes to Clojars CDN setup). Once that's fixed, I'll rebuild the clj-new docs.#2020-02-1402:01rbergerThanks!#2020-02-1313:25Andreas S.Hello I try to run clj on windows in cygwin but I get:#2020-02-1313:25Andreas S.$ clojure
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main#2020-02-1313:25Andreas S.Any ideas?#2020-02-1313:33delaguardotry clojure -Spath you should get a string with all the jars including clojure-%version%.jar then go check if this jar is actualy available#2020-02-1313:36Andreas S.If I run "clojure -Spath" I get the same error#2020-02-1313:36borkdude@andreas.scheinert We had someone here a while ago who has an older version of clojure on his path. What do you see when you type clojure -h#2020-02-1313:37Andreas S.I have a file here: /usr/local/lib/clojure/libexec ==> clojure-tools-1.10.1.507.jar*#2020-02-1313:38Andreas S.borkdude here is version "clojure -h" print out#2020-02-1313:38Andreas S.Version: 1.10.1.507#2020-02-1313:38borkdudethat's good then#2020-02-1313:40Andreas S.I know its a bit side stepping but, ok I took the effort and installed it on powershell#2020-02-1313:40Andreas S.clj and clojure give me now a REPL#2020-02-1313:40Andreas S.but#2020-02-1313:41Andreas S.how do I run the deps code from here in powershell? https://github.com/clj-python/clj-template#2020-02-1313:41Andreas S.clj -Sdeps '{:deps {seancorfield/clj-new {:mvn/version "0.8.6"}}}' \
-m clj-new.create \
\
appcompany.funapp
#2020-02-1313:42Andreas S.I figured I remove all the \ and create a long single line of command, but then that still gives me:#2020-02-1313:42Andreas S.Error while parsing option "--config-data {:deps {seancorfield/clj-new {:mvn/version 0.8.6}}}": java.lang.NumberFormatException: Invalid number: 0.8.6#2020-02-1317:36dominicmI notice that resolve-deps doesn't mention that it will be concurrent even if :threads isn't provided. Should it?#2020-02-1317:36seancorfieldAh, follow this guide for Windows https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#escaping-quotes#2020-02-1317:40seancorfieldEven following that Windows guide didn't work for me 😞#2020-02-1317:41dominicmmake-classpath-roots mentions the system separator, but it doesn't really make sense in the context.#2020-02-1317:42seancorfield@andreas.scheinert This worked for me:
> clj -Sdeps '{:deps {seancorfield/clj-new {:mvn/version ""0.8.6""}}}' -m clj-new.create appcompany.funapp
So, double "" inside '#2020-02-1317:47seancorfield(that said, even with Python 3.7 installed from the Windows Store, I can't actually get the new project to run but that should be taken up in #libpython-clj at this point)#2020-02-1411:42Andreas S.Thank you sean for replying! Yes the quoting did the trick but I am stuck kat the same point as you I couldn't get windows python to work with libpython-clj#2020-02-1320:33samFor private git repo deps on Windows, is only Pageant supported as the ssh agent for authentication or can OpenSSH also work?#2020-02-1321:03alexmillertry it and see?#2020-02-1321:03alexmillerI know that pageant works, don't know about openssh#2020-02-1321:13sam@alexmiller I have been trying. It seems to be attempting to use OpenSSH. I can tell because the specific error message I get changes depending on whether github is added to .ssh/known_hosts. However, even though I can use git on Powershell to clone a private git repo, I get an error when trying to run clj with a private github dep. The lowest part of the stack trace is just:
Caused by: com.jcraft.jsch.JSchException: Auth fail
at com.jcraft.jsch.Session.connect(Session.java:519)
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:126)
... 45 more
which doesn’t give me a ton to go on. It’s been really painful to debug.#2020-02-1321:13alexmillerwell I think I would say, no it doesn't work then#2020-02-1321:14samIt’s been finicky to get it working even with Pageant. So I’m never sure whether something just doesn’t work, or if I’m just doing something slightly wrong#2020-02-1321:14alexmillerssh is generally tricky, particularly with access to the wide world of credential providers#2020-02-1321:16alexmillerthere are some options (including shelling out to just use git) which we have explored. newer versions of jgit use apache mina instead of jsch. I have spent some time working on evaluating these alternatives but haven't had time to really get to the end of that process#2020-02-1321:16alexmilleranother option would be to clone, then use as a local repo instead#2020-02-1321:17alexmilleror to clone, build and install a local artifact, then use that#2020-02-1321:20samOkay, thanks. One other question for you. I’ve resorted to trying OpenSSH on one of our dev’s windows machines only because we couldn’t get deps working with Pageant on that machine. I have no idea why it’s not working because we’ve been able to get it working on other Windows machines. Based on your experience, do you have any idea of any “gotchas” we might be running into? The error message is the same as above.#2020-02-1321:21samIf not, that’s okay, I know that’s a very vague question. Just wondering if anything comes to mind of the top of your head#2020-02-1321:22alexmillerI don't unfortunately. the jsch stuff is a bit flaky#2020-02-1321:26samOkay, well thanks anyway. Btw, just wanted to say I was at StrangeLoop in September and really enjoyed it. Thanks for doing that.#2020-02-1321:27alexmillergreat! thx#2020-02-1321:24ghadiif you're using any ssh agent, make sure the SSH_AUTH_SOCK variable is set so that Clojure (via JSch) knows where the agent is#2020-02-1321:27samcool, thank you, I’ll look into that.#2020-02-1322:01samAre you sure this is applicable for Windows? I can’t find much information about it in the context of Windows, and Clojure seems to work fine on my windows machine using Pageant as the ssh-agent without this env variable being set.#2020-02-1322:03ghadissh clients discover where an agent is running by consulting that environment variable#2020-02-1322:04ghadithat's what jsch does, too.... sorry Windows just adds a lot of extra wildcards because of all the various linux-y environments#2020-02-1322:04ghadipaging @U04V70XH6#2020-02-1322:05ghadijust in case he knows#2020-02-1322:11seancorfieldI know nothing about ssh on windows, sorry #2020-02-1322:12seancorfieldI only ever use ssh stuff via Ubuntu / WSL on windows 😁#2020-02-1322:15samThanks anyway!#2020-02-1400:30andy.fingerhutI've added one section to the README for my haironfire repo here: https://github.com/jafingerhut/haironfire#some-leiningen-projects-can-depend-upon-multiple-files-andor-tools-to-determine-the-actual-dependencies. merely mentioning some challenges of calculating accurate dependencies for Leiningen projects, which I have not attempted to quantify in my work how often those tricky cases even arise in the projects I've analyzed. I don't have any plans to try to analyze those things, but others are certainly welcome to if they are interested, using any or all of the code in that repo if it helps. Perhaps the biggest hole in the 91.8% and 95.6% numbers I reported for EDN-readable project.clj files are: (a) Those were for repos with a project.clj file in the home directory. Outside of the 100% number is about another 10% of repos that have no project.clj nor deps.edn file in the home directory, many of which probably contain project.clj files in subdirectories that I haven't attempted to find or read. And (b) the difficulties in determining dependencies even after you read a single project.clj file, if there are tools like Lein-modules or similar things in use, again which I haven't tried to determine which projects might use such things.#2020-02-1400:32andy.fingerhutOr in summary, I will now more fully understand if the decision is made not to have tools.deps read project.clj files 🙂#2020-02-1407:33dominicmSomething just surprised me while I was writing up a repro for 132, this dependency tree resolves without error, because somehow B and B compare as the same version, but I don't understand how because there's no compare-version for :local :local (yet, that's what my patch does).
deps.edn
+--A {:local/root "/A"}
| |
| +- B {:local/root "/B"}
+--C {:local/root "/C"}
|
+- B {:local/root "/B"}
It doesn't when there's a git in the way for A/C. I think the cause is https://github.com/clojure/tools.deps.alpha/blob/1feb495adfc5c72c9c5fc9a64917a6bab2203c1a/src/main/clojure/clojure/tools/deps/alpha.clj#L149 not using (= 0 (compare-version a c)) .#2020-02-1407:53dominicmNothing to do with git, this equivalence comparison just already exists. So my changes don't help. I could have sworn I tested it...#2020-02-1407:54dominicmOh, I did. But I was being very, very stupid (= (:local/root coord-x) (:local/root coord-x)) comparing x to x 🙂#2020-02-1418:42dominicmI wrote a project to visualize the relationship between your project and it's dependencies. It currently relies on the deps.edn format in order to function. Here's the project: https://github.com/SevereOverfl0w/vizns#2020-02-1418:42dominicmI wrote a project to visualize the relationship between your project and it's dependencies. It currently relies on the deps.edn format in order to function. Here's the project: https://github.com/SevereOverfl0w/vizns#2020-02-1419:22andy.fingerhutLooks like a useful thing to add somewhere on this wiki page, if it is not already there: https://github.com/clojure/tools.deps.alpha/wiki/Tools#2020-02-1419:33borkdudedone#2020-02-1419:54dominicmThank you 😚#2020-02-1711:33teodorluThis is really cool! Fantastic way to confirm that your code is all spaghetti after all.
Some quick notes on usability:
• I first naïvely ran it in a folder without a deps.edn file, and spent some time figuring out why it didn't work.
• Then I reran it in an actual project, only to find out I was making it crash because I hadn't supplied any main args
• Then I figured I needed to sudo apt install graphviz, because of the dependency on dot.
Then all was fine!#2020-02-1712:21dominicmI don't have the headspace to pick those up yet, but I'd welcome a PR if you have time. Thanks for the feedback, I appreciate it!#2020-02-1712:26teodorluI created https://github.com/SevereOverfl0w/vizns/issues/4 🙂#2020-02-1420:14alexmiller@dominicm the readme example has tools.deps.graph for the main class#2020-02-1420:36dominicmD'oh, used that as inspiration. I thought I'd got it all, will fix, thanks.#2020-02-1420:16alexmillerI assume that should be io.dominic.vizns.core#2020-02-1420:18alexmilleralso, it looks like you happened to rely on make-classpath-roots which I added in tda 0.8.666 (which I never announced) then made public in the release earlier today#2020-02-1420:18alexmillerI didn't expect anyone would make use of it in interim :)#2020-02-1420:19alexmillerseems like you could have just used make-classpath though?#2020-02-1420:49dominicmI could have split the string, yes. It was nice to think of all the saved cycles of not doing that.#2020-02-1420:52alexmillerI sent you a pr#2020-02-1421:00dominicmOh, that's a better solution. Thanks for the PRs, makes it mergeable from my phone. The final one won't merge at the moment, but I can resolve from a keyboard :)#2020-02-1420:20alexmillerthere's some ongoing discussion between rich and I in that area about structured classpath data and I'm not ready to commit to anything there yet#2020-02-1420:21borkdudemaybe the tool should just deal with a classpath as input so users can decide which tool they use to create it (`clojure -Spath`, lein classpath, etc)?#2020-02-1420:50dominicmI want to do that, but then I have to identify classpath entries back to their library name.#2020-02-1420:51dominicmI'd need something slightly different to a classpath. Haven't figured out what that looks like yet from an agnostic point of view.#2020-02-1420:55dominicmGit dependencies are pretty difficult to name.#2020-02-1421:04borkduderight!#2020-02-1420:21borkdude(I haven't looked at any details so I don't know if that's going to make sense)#2020-02-1420:32borkdudefor the project.clj git dep problem, one maybe silly but effective solution could be to let tools.deps clone the project, then let it invoke lein classpath on that project, so lein can deal with itself, and write that classpath to a file which is cached, so next time tools.deps wants to know the classpath that that dep is going to produce, it doesn't have to invoke lein again#2020-02-1420:33borkdudehaven't really thought this through, feel free to dismiss it. have a great weekend. 🍻#2020-02-1420:35alexmillerI think it's interesting to consider a dep type that could provide it's deps explicitly like that#2020-02-1420:35alexmillerhadn't thought about it before#2020-02-1420:48andy.fingerhutThat still requires eval-level trust in projects you depend upon, at dep resolution time, yes? I do not know if it is any kind of clear bright line to avoid that -- just asking to confirm that the proposed idea still does do eval of project.clj files, even if you never require their code.#2020-02-1421:02borkdudetrue, it will eval. it could potentially be a solution for lein projects with java sources as well btw.#2020-02-1421:04borkdudeI guess the user of a git dep can check the project at given sha if it's deemed safe#2020-02-1421:06borkdudeRunning library code from the wild is going to be eval'ed anyway, so you should in theory always be checking the code of a dep before you run it?#2020-02-1421:15andy.fingerhutIt would be safest, yes. There is a lot of web of trust kinds of things going on, for sure, and the common practice that if something is discovered bad, it will be publicized. That all works with your idea, too, still. I'm only perhaps nit-picking that deps resolution time is another eval-time opening this creates, over and above require'ing the code and running it. To take advantage of it, a malicious party still needs to become trusted, i.e. depended upon by other projects, so in that sense it isn't a new opportunity for attack.#2020-02-1509:38penryuI'm used to seeing maven dep versions in deps.edn as {:mvn/version "0.4.490"}}, but I've found one that specifies deps with #:mvn{:version "0.4.490"}. Looks like a reader macro, but I haven't seen it before.#2020-02-1509:39borkdudethat's called a namespaced map#2020-02-1509:40penryuit looks like it's equivalent to the other syntax; are they interchangeable?#2020-02-1509:40borkdudesee "Map namespace syntax" here: https://clojure.org/reference/reader#_map_namespace_syntax
it's supported since 1.9 and yes, they are interchangeable#2020-02-1509:41penryuAh, you can use it to factor out a common namespace in map keys?#2020-02-1509:43penryuThat's exactly what I was missing. Thanks, @borkdude.#2020-02-1517:08borkdudesince plk is a kind of port of the clojure bash script and is listed here: https://github.com/clojure/tools.deps.alpha/wiki/Tools#runtimes
would it make sense to add deps.clj there as a port of the clojure bash script to clojure itself?#2020-02-1517:09alexmillerYep#2020-02-1517:10borkdudedone#2020-02-1517:11borkdudeI was just informed of this project: https://github.com/benedekfazekas/morpheus#2020-02-1517:11borkdudeMight make sense to add here: https://github.com/clojure/tools.deps.alpha/wiki/Tools#deps-management, since that also lists other viz tools (cc @benedek)#2020-02-1517:13benedekdone#2020-02-1611:07kwladyka~/L/M/c/s/P/l/pdf-converter clj -Aoutdated
| Dependency | Current | Latest |
|---------------------------+----------+----------|
| org.clojure/clojure | 1.10.0 | 1.10.1 |
| org.clojure/clojurescript | 1.10.520 | 1.10.597 |
! ~/L/M/c/s/P/l/pdf-converter clj -Aoutdated -u
Updating: deps.edn
org.clojure/clojure {:mvn/version "1.10.0"} -> {:mvn/version "1.10.1"}
org.clojure/test.check {:mvn/version "0.9.0"} -> {:mvn/version "0.10.0"}
com.cognitect/test-runner {:sha "028a6d41ac9ac5d5c405dfc38e4da6b4cc1255d5"} -> {:sha "f7ef16dc3b8332b0d77bc0274578ad5270fbfedd"}
~/L/M/c/s/P/l/pdf-converter clj -Aoutdated
| Dependency | Current | Latest |
|---------------------------+----------+----------|
| org.clojure/clojurescript | 1.10.520 | 1.10.597 |
~/L/M/c/s/P/l/pdf-converter #2020-02-1611:08kwladykaHow https://github.com/Olical/depot work?#2020-02-1611:08kwladykaWhat is the logic here?#2020-02-1611:08kwladyka{:paths ["resources" "src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}}
:aliases {:test {:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}
me.raynes/fs {:mvn/version "1.4.6"}}}}
:runner {:extra-deps {com.cognitect/test-runner
{:git/url ""
:sha "f7ef16dc3b8332b0d77bc0274578ad5270fbfedd"}}
:main-opts ["-m" "cognitect.test-runner"
"-d" "test"]}}
deps.edn#2020-02-1611:16kwladykait is showing something different to update and update something different#2020-02-1611:16kwladykaand what cljs doing here?#2020-02-1611:37dominicmMaybe the first one is looking at transitive dependencies when it shouldn't be?#2020-02-1611:38kwladykaWhat do you use for updates? I found there is also vulcan#2020-02-1611:46dominicmI don't use anything :)#2020-02-1611:49borkdude@kwladyka I'm not using it myself, but this hacky script also works: https://github.com/borkdude/babashka/#list-outdated-maven-dependencies#2020-02-1611:51borkdude(for maven deps)#2020-02-1618:17seancorfieldI would never run a tool that auto-updated my deps. I used Depot for a while but the 2.x version doesn't work with our setup (because it processes deps files directly instead of relying on t.d.a. to process all the aliases and override etc via a non-user deps.edn specified via CLJ_CONFIG).#2020-02-1618:19seancorfieldLike @borkdude I created a script instead -- it runs clojure -Stree and massages the output, then creates a deps.edn with all the :mvn/version deps changed to "RELEASE" and re-run clojure -Stree against that, and then compare the top N levels of those trees.#2020-02-1710:00Michaël SalihiThis script is available on your Github ?#2020-02-1710:01borkdude@UFBL6R4P3 There is one variation available here:
https://github.com/borkdude/babashka/#list-outdated-maven-dependencies#2020-02-1710:02borkdudeI made it based on the idea of Sean#2020-02-1710:17Michaël Salihi@borkdudeNice script, to use and also for learning beautiful Clojure scripting.
Babashka rocks! Thank you!#2020-02-1710:23borkdudeThanks 😊#2020-02-1717:56seancorfield@UFBL6R4P3 Our script at work is somewhat proprietary and heavily tied to how our monorepo works (30 subprojects, each with their own deps.edn and with an overall deps.edn that is used instead of the ~/.clojure user-level one, via the CLJ_CONFIG environment variable).#2020-02-1718:45Michaël Salihi@U04V70XH6 I understand. Thx for the information, I don't know CLJ_CONFIG.
New thing in my baggage!#2020-02-1707:13benedeka tool that creates a PR automatically would be nice i think. added benefit is all the tests run if you have some autobuild set up. such tools exist for other ecosystems i think#2020-02-1910:08souenzzoFrom
https://github.com/portkey-cloud/cfn-yaml/issues/1
How I supposed to import a project withtout a clear ArtifactID?
Consider:
I publish my/lib {:rev :v1} that requires cfn-yaml {:git/url "" :sha v1}
Someone else import my/lib {:rev :v1} portkey-cloud/cfn-yaml {:git/url "" :sha v2}
It will result in git deps.#2020-02-1915:03alexmillerit kind of is with the advice of
github-yourname/time-lib
#2020-02-1914:15dominicmIf repos were the canonical source of what id they represented, that would solve this. It would also make the deps.edn format very different.#2020-02-1914:18alexmillerit would also break forking use cases etc#2020-02-1914:19alexmillerit was long considered#2020-02-2108:50teodorluHey! 👋
Any idea on why this doesn't work?
$ clj -e '(doc ns-map)'
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: doc in this context
Full report at:
/tmp/clojure-135646063998456412.edn
#2020-02-2108:58dominicmYou need to require clojure.repl, doc is defined there, not in core#2020-02-2108:59dominicmuser.clj requires clojure.repl for you#2020-02-2113:54alexmillerNo, Clojure.main does this in the repl setup#2020-02-2109:00vlaaadI would guess it would be more correct to say that starting repl requires clojure.repl for you#2020-02-2109:03vlaaadI would guess even more correct thing to say that clojure.main/repl-requires are required when you start a repl https://github.com/clojure/clojure/blob/38524061dcb14c598c239be87184b3378ffc5bac/src/clj/clojure/main.clj#L355#2020-02-2109:06dominicmQuite right, I'm not sure why I thought otherwise. Brain fart.#2020-02-2113:14Wes HallMaking some increasingly half-hearted attempt to get DynamoDBLocal running to start embedded for tests, but it seems to depend on sqllite4java which exists in the mvn repos but with <packaging>dylib</packaging>. Am I right in assuming that tools-deps doesn't have support for downloading dylib dependencies?#2020-02-2113:48kszaboI prefer to use docker-compose to set up local service dependencies:
version: '2'
services:
dynamodb-local:
image: 'dwmkerr/dynamodb'#2020-02-2113:48kszabohttps://hub.docker.com/r/dwmkerr/dynamodb#2020-02-2113:49kszabobut this is not related to #tools-deps this way#2020-02-2114:20Wes Hall@thenonameguy Yeah, I will probably move to something like that. Dynamo local, at least in theory, does support embedded running so should be possible to throw up an in-memory version in a text fixture and tear it down afterwards, but the dependency on sqllite (and thus native libraries), is a bit of a hurdle. I'll probably give up, but just thought I would check that I am not missing a route I could at least try, but I think the docker-compose option is probably the way to go. Thanks.#2020-02-2114:21Wes HallDownloading native (thus platform dependent) libraries, is a bit awful anyway, I wouldn't be surprised if it's not supported.#2020-02-2114:33teodorlu@vlaaad @dominicm @alexmiller thanks for the help clearing that up! I realize why doc shouldn't work above, and I get the expected result when I require it as I "should" do.
$ clj -e "(require '[clojure.repl :refer [doc]]) (doc ns-map)"
-------------------------
clojure.core/ns-map
([ns])
Returns a map of all the mappings for the namespace.
#2020-02-2117:14rickmoynihanNot sure if this is a known regression or not… but I was using some private S3 buckets as mvn repos; with a URL of the form s3p://<bucket>/<directory>/
I’ve been using this for years (previously with leiningen) and found earlier in the week that I could no longer resolve deps via the clj tool.
I suspect this was caused by the recent changes in t.d.a to using the cognitect aws lib; but I can’t be certain.
Anyway I’ve just tried changing the bucket URL’s to be of the form s3:// and it now works. No idea what the difference is in the protocols; but I’ve seen both forms over the years.#2020-02-2117:17gerreds3p is for hadoop/presto it seems.#2020-02-2117:18gerrednot an official protocol. AWS has a few other protocols for similar reasons (with EMR specially): s3a and s3n come to mind.#2020-02-2117:18gerrednot an official protocol. AWS has a few other protocols for similar reasons (with EMR specially): s3a and s3n come to mind.#2020-02-2117:19gerredhttps://github.com/s3-wagon-private/s3-wagon-private i see this of course#2020-02-2117:19gerredso maybe this was only a wagon thing.#2020-02-2117:21gerredit looks like it's even gone from newer versions of Wagon#2020-02-2117:22rickmoynihanyeah iirc tools.deps used the s3 wagon until recently#2020-02-2117:23gerredoh no it's still there#2020-02-2117:23gerredhttps://github.com/s3-wagon-private/s3-wagon-private/blob/master/src/main/java/org/springframework/build/aws/maven/PrivateS3Wagon.java#2020-02-2117:23gerredyeah#2020-02-2117:27dominicmAlex did ask to know if anything broke. I think compatibility was the intention.#2020-02-2117:33rickmoynihanYeah that’s why I’m mentioning it.
The (end user) fix is trivial (just change to use s3://) once you know what the problem is though.#2020-02-2117:53alexmillers3p has never been a documented supported protocol for clj#2020-02-2117:54alexmillerSo was working before by accident, not by intent#2020-02-2117:54alexmillers3p is an invention of the spring aws wagon#2020-02-2117:55rickmoynihanAh interesting, I think I just copied my URLs from lein and didn’t think anything of it when they worked#2020-02-2117:55rickmoynihanAnyway it’s not a problem, I just thought I’d mention it incase it happens to others#2020-02-2405:54onetomI'm trying to switch to Google's Maven Central mirror.
I've added a mirror section to my ~/.m2/settings.xml but I'm not sure if it's used by clj.
How can I check what actual URL is it downloading from when it says:
Downloading: org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar from central
I've tried -Sverbose and -Sdescribe but neither of those options provide an answer#2020-02-2413:52alexmillerI don’t think there is any way to see that, but there is code with the intent to use mirrors in settings#2020-02-2413:52alexmillerSomeone else from Asia walked down this same path a few months ago and it was working#2020-02-2406:00EmmanuelOgaperhaps you need to also add it to the deps.edn map?#2020-02-2406:00EmmanuelOgahttps://clojure.org/reference/deps_and_cli#_procurers#2020-02-2406:01onetomI'm trying to move it out from my deps.edns becaues mirror settings are not project specific#2020-02-2406:02EmmanuelOgawhat if you put it in the global deps.edn?#2020-02-2406:02EmmanuelOga$HOME/.clojure/deps.edn#2020-02-2406:04onetomthen pure maven tools wouldn't benefit from this setting#2020-02-2406:04onetomplus im not sure what's the semantic difference between overriding the repo called "central" vs specifying mirrors instead#2020-02-2406:06onetomit's also a bit unclear how the :mvn/repos key is merged into the hardwired default values, because i haven't found a way to print the effective value of it using clj#2020-02-2406:08onetomi had the impression that the :mvn/repos key in my project's deps.edn overrides the value coming from ~/.clojure/deps.edn but i haven't found this being documented#2020-02-2406:14onetomthe only reason i suspect that the <mirror> setting is taken into account is because the installation of my deps from scratch takes 34s instead of 51s#2020-02-2406:34EmmanuelOgaheh nice#2020-02-2406:36EmmanuelOgaare you working with a setup that for some reason needs toperform installs from scratch often?#2020-02-2407:02onetomim in hong kong and im on crappy mobile connections often, so using mirrors make a big difference even if im not doing from-scratch installs.
on the other hand im also experimenting with building self-contained developer workstation setups and then i want to test from-scratch builds sometimes just to make sure everything still works.
a datomic upgrade also pulls in quite some dependencies and our projects might be on slightly different versions of common libraries and all this adds up...
just to demonstrate the absolute scale of problems:
• im taking a 30minute ferry ride twice a day to work and i could spend about 20minutes of that working
• i might need to run a nix package or maven update every week probably which would amount to downloading ~200MB. that can take 10minutes; half of my travel time is wasted.
• if i use a mirror and it just takes 5 minutes to upgrade, only 1/4th of my travel time is wasted.
but generally many things are significantly slower from asia, so even if i run something on a singapore/tokyo/hong kong aws or digital ocean server, the download time differences add up to minutes compared to US servers.#2020-02-2407:03onetomif u consider that multiple ppl in the company suffer that time waste from time to time, then it's a non-negligible factor#2020-02-2407:05onetomand then i haven't even talked about ppl in china-proper, behind the GFW, who often have to go thru very slow vpns.
for them an asian mirror might be several times faster or not even being blocked, so they dont need a vpn to access...#2020-02-2407:07EmmanuelOgaoh wow ok setting up that mirro rsounds like a worthy goal#2020-02-2407:07onetombecause of these circumstances, im very excited to see parallel downloads in tools.deps!
now it performs just as fast as npm2/yarn/pnpm for example 🙂#2020-02-2407:07onetomone less critique i have to listen to from former JS ppl whom im trying to teach clojure to...#2020-02-2407:09EmmanuelOgawould it make sense to setup some sort of system wide caching proxy? quick google found this thing: http://proxychains.sourceforge.net/#2020-02-2407:10EmmanuelOgaI imagine there could be a way to make it so every time you open an http connection in any program on your system it would go through your proxy#2020-02-2407:10EmmanuelOgaI'm not very good at this kind of thing though so maybe it is just a stupid idea hah#2020-02-2407:11onetomproxychains doesnt do caching, but yeah, it would make sense to setup some proxies, but every ecosystem needs different kind of proxies and then some of the deps we need are private because they are our own, in-house code, or it's datomic.
setting up proxy for such an environment is not trivial at all.#2020-02-2407:11EmmanuelOgagotcha#2020-02-2407:12onetomive experimented with various maven mirroring solution, like jfrog and artifactory, etc, but that doesn't solve brew, macos, git, nix, npm caching...#2020-02-2407:16onetomfor proxychains, i've actually made a nix package:
{ stdenv, fetchFromGitHub } :
stdenv.mkDerivation rec {
pname = "proxychains";
version = "4.14";
src = fetchFromGitHub {
owner = "rofl0r";
repo = "proxychains-ng";
rev = "v${version}";
sha256 = "03wk2xpxwc7kwlq6z9jf9df1vlh6p0dn0kf826fh1k7nfaa8c4py";
};
postPatch = ''
# Temporary work-around; most likely fixed by next upstream release
#sed -i Makefile -e '/-lpthread/a LDFLAGS+=-ldl'
'';
meta = {
description = "Proxifier for SOCKS proxies";
homepage = ;
license = stdenv.lib.licenses.gpl2Plus;
platforms = stdenv.lib.platforms.all;
};
}
and using this config file for it:
[ProxyList]
socks5 127.0.0.1 1080
u can just ssh -D 1080 some.aws.basition.host to establish a tunnel, then
export MYSQL_HOST=
proxychains4 -f proxychains.conf mysql --host=$MYSQL_HOST --user=root --password -e 'show databases'
would allow accessing an RDS instance using its internal aws hostname, although the mysql cli tool doesn't support socks proxies.
just showing, to clarify what proxychains is for#2020-02-2407:19onetomalso, this kind of proxying over SSH is also said to be disrupted across the GFW.
i find it a bit hard to believe, but that's what i've heard...#2020-02-2407:21onetomwhile all this might sound off-topic on this channel, i think it's actually not, because these kind of scenarios are the reasons, why supporting various directives in maven settings.xml or providing parallel downloads is important.
it's just ppl suffering from these issues don't really have a voice to raise them, because they don't speak good english and they are not aware where should they speak up...#2020-02-2407:24EmmanuelOgawould you say clojure is popular in hong kong?#2020-02-2407:24onetomwell, im personally making sure it is ;D#2020-02-2407:26EmmanuelOganice. Good luck with that 🙂#2020-02-2407:26onetomi've introduced clojure into almost all companies i've worked at in the past ~5years https://www.linkedin.com/in/onetom/
im not going to meetups often lately, but when i do, i also sow the seeds of clojure amongst the audience 😉#2020-02-2407:27EmmanuelOgado you speak chinese and/or cantonese?#2020-02-2407:28onetomnope, only a bit of thai, but not at a technical level; im hungarian.
i've started to shake up the hungarian clojure community lately too on #clojure-hungary for example#2020-02-2407:29EmmanuelOgacool#2020-02-2601:34cflemingAm I correct in thinking that this commit is what’s needed to read the authentication for a particular repo from settings.xml? https://github.com/clojure/tools.deps.alpha/commit/658074337ce9577c9f1298fd5146a3029308748c#2020-02-2601:35alexmillerYes, modulo any later changes#2020-02-2601:36cflemingOk, I’ll look at that. Seems pretty magic in there. It’s odd that that functionality isn’t part of the resolver itself, it seems to live in a Maven jar.#2020-02-2601:36alexmillerDon’t get me started#2020-02-2608:03dominicmI think I ported something I found in the maven cli code. Was a bit of a hunt.#2020-02-2620:29neupshHello, I have @seancorfield’s deps.edn file from https://github.com/seancorfield/dot-clojure in my ~/.clojure/deps.edn and I am running clj -A:rebel:deps to start a clojure repl with rebel-readline and tools.deps.alpha for adding libraries from within the repl. So, to be able to add library, I will have to (require '[clojure.tools.deps.alpha.repl :refer [add-lib]]) before I can call add-lib. Is there a way to auto require this in the user namespace so that when I start the repl, I have it available there?#2020-02-2620:49seancorfield@neupsh You have to split the resource/main opts:
> clj -R:rebel:deps -e "(require '[clojure.tools.deps.alpha.repl :refer [add-lib]])" -m rebel-readline.main
[Rebel readline] Type :repl/help for online help info
user=> (add-lib 'seancorfield/next.jdbc {:mvn/version "RELEASE"})
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Downloading: seancorfield/next.jdbc/maven-metadata.xml from
true#2020-02-2620:50seancorfieldYou could add a new alias that combines all of that -- you'll need , instead of whitespace in the -e option if you put it in deps.edn tho'.#2020-03-0122:50samI wrote a script to aot-compile classes in my project, which I have configured to run automatically when necessary via aliases. e.g.
:test {:extra-paths ["test" "test-resources"]
:extra-deps {com.cognitect/test-runner {:git/url ""
:sha "209b64504cb3bd3b99ecfec7937b358a879f55c1"}}
:main-opts ["-i" "aot.clj"
"-m" "cognitect.test-runner"]}
I tried adding logging to my aot script using clojure.tools.logging, and found that tests started failing with java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
I don’t totally understand what’s going on, but I think it has something to do with the fact that the clojure compile function transitively compiles only namespaces that haven’t already been loaded. So, because I have already loaded clojure.tools.logging before calling compile, it does not get transitively compiled even though the namespaces I’m compiling depend on it.
Still, I wasn’t sure that this situation should cause a NoClassDefFoundError, since the clojure source code should still be available on my classpath, but I think it might be due to this issue: https://clojure.atlassian.net/browse/CLJ-1544. If I understand correctly, I think it’s important in my case for clojure.tools.logging to be compiled since the calling code is compiled.
My workaround was to add an explicit (compile 'clojure.tools.logging) to the script.
So my questions are:
1. Does my understanding of the problem make sense?
2. Is there an issue here with compile that should be addressed? Assuming my understanding of all this is correct, I wish compile had options about how it should handle dependency namespaces, or at least that the documentation specified this behavior (if it does, I couldn’t find it).#2020-03-0123:09seancorfield@samwagg0583 AOT has a number of serious "gotchas" and it's best to avoid it unless you absolutely need it (it's "safe" as the very last step before you build an uberjar for deployment as a complete application, but most other uses can trip you up).#2020-03-0123:10alexmillerI don’t think this is CLJ-1544#2020-03-0123:11alexmillerCompile is transitive but it can’t necessarily “see” everything and you may need to explicitly compile things sometimes. All that is normal.#2020-03-0123:12seancorfieldWhen code is (partially) AOT'd and then it attempts to load in (and compile on-demand) more code, you can get mismatches between the classes that the class loaders can see. Is that a fair description @alexmiller?#2020-03-0123:14alexmillerIt’s more specific than that#2020-03-0123:15alexmillerSorry, I gotta run #2020-03-0123:13alexmillerYour issue with having it already loaded is something to consider and you may need to change your compilation process to account for that#2020-03-0123:17seancorfield@samwagg0583 What is driving your need to AOT code for testing here?#2020-03-0123:58samWe are using a java library for distributed data pipelines called Apache Beam and running in Google Cloud Dataflow. Running a pipeline involves running code on a local JVM that uploads serialized java objects to remote JVMs. De-serialization on the other end requires class files, hence the AOT.#2020-03-0123:59samMy workaround of explicitly compiling classes that are already loaded seems to work okay. I just wanted to better understand the issue.#2020-03-0200:06seancorfieldOuch. Yeah, you don't have much of a choice I guess. Tho' I thought I'd read about a Beam-related library that avoided AOT recently?#2020-03-0200:06seancorfieldYeah https://github.com/atdixon/thurber#2020-03-0202:59samThanks for this! We have looked into existing clojure libraries for Beam in the past, but I wasn’t aware of this one. I’ll definitely have to check it out further.#2020-03-0203:33seancorfieldIt's pretty new. I can certainly understand the authors' desire to avoid AOT for this 🙂#2020-03-0214:16sparkofreasonI've been using thurber, so far very successfully. Aaron has also been very responsive to issues and questions. It's still a bit raw, but useable, and I like the philosophy of keeping Clojure wrappers of Java APIs as thin as possible. Makes it a lot easier to figure out how to go from Java-based docs etc. to the Clojure equivalent.#2020-03-0217:33aviTIL!#2020-03-0200:07seancorfield> AOT Nothing
Fully dynamic experience. Reload namespaces at whim. thurber's dependency on Beam, Clojure, etc versions are completely dynamic/floatable. No forced AOT'd dependencies, Etc.#2020-03-0219:20fabraohello all, using deps.edn is there any way to load dependency dynamic as boot does?#2020-03-0219:20ghadiwhat does that mean @fabrao?#2020-03-0219:20borkdudehe might be looking for the add-lib stuff#2020-03-0219:20borkdudehttps://insideclojure.org/2018/05/04/add-lib/#2020-03-0219:21fabraoHumm, I think it´s what is I want, let me check#2020-03-0219:22borkdudeit is in a branch, not officially out yet, but might be later#2020-03-0219:24fabraoso, for now only include new lib in :deps to include new lib and restart repl?#2020-03-0219:28seancorfield@fabrao My dot-clojure file has an alias that brings in a recent add-lib branch of tools.deps.alpha and notes on how to use it to dynamically load new libraries into your REPL without a restart.#2020-03-0219:29seancorfieldhttps://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L144-L147#2020-03-0219:30seancorfieldI use that quite a bit to load extra libraries while I'm working.#2020-03-0219:31seancorfieldYou can see how I use it with next.jdbc to bring in all of its test dependencies into the REPL while I'm working on other projects https://github.com/seancorfield/next-jdbc/blob/master/test/next/jdbc/test_fixtures.clj#L133-L149#2020-03-0219:35fabraoI have to change it to use with cider#2020-03-0411:14practicalliTo use a deps.edn alias with Cider (Emacs), either use the universal constant to edit the jack-in command or create a .dir-local.el file and set the cli to run an alias
https://practicalli.github.io/blog/posts/cider-jack-in-to-clojure-cli-projects-from-spacemacs/#2020-03-0219:35fabraoin emacs#2020-03-0219:50fabraoit seems that it works only in REBL enviroment#2020-03-0219:55seancorfieldNo, it works in a plain REPL.#2020-03-0219:56seancorfield(in fact, it specifically does not work in REBL because REBL does not use a DynamicClassLoader)#2020-03-0219:57seancorfieldYou don't need to change anything -- it's pure Clojure and works with any plain REPL, as long as you had the add-lib branch dependency on your classpath when you start the REPL.#2020-03-0219:58seancorfieldI just start a REPL with clj -A:deps (and other options for my use). If you're using Leiningen, you'd need something that pulls in Git dependencies (I do not use Leiningen).#2020-03-0220:04alexmillerI thought we fixed the rebl thing?#2020-03-0220:04alexmillermaybe I just filed tickets to fix it generically in prepl and socket repl#2020-03-0220:05alexmillerdon't remember now#2020-03-0220:05ghadiyeah tickets filed for DCL#2020-03-0220:05ghadione ticket, I think#2020-03-0220:05alexmilleryeah#2020-03-0220:05ghadihttps://clojure.atlassian.net/projects/CLJ/issues/CLJ-2540#2020-03-0220:06alexmilleryeah, it's in the chute for 1.11#2020-03-0312:21geraldodevhow we can programatically get the list of loaded aliases ?#2020-03-0312:27vlaaad(.getAliases *ns*)#2020-03-0312:38vlaaadoh, sorry, this is tools-deps, I thought you are asking about namespaces#2020-03-0312:39borkdudeThis would be the public API for that I think (I know, this isn't about tools.deps):
user=> (require '[clojure.string :as str])
nil
user=> (ns-aliases *ns*)
{str #object[clojure.lang.Namespace 0x301d8120 "clojure.string"]}#2020-03-0312:41vlaaadah, right, forgot about that one 😄#2020-03-0312:53vlaaadback to original question @U0516053R :
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "dcacaeb3aabd9ce5e5f1eae9eae8ee"}, :content ("[email protected]")}#2020-03-0312:54vlaaadI don’t think there is a good and reliable way to figure it out, because clj launches a new process with classpath, so no alias information is really available to the jvm#2020-03-0312:55vlaaadwith that said, you can lookup your PID, lookup parent PID, run ps (linux and mac only) to find information about parent PID and extract some info from that: as you see, :out contains invocation string#2020-03-0312:59vlaaadnote: ProcessHandle is available since JDK 9, previous JDKs will require some reflection and googling to get pids..#2020-03-0313:06geraldodev@U47G49KHQ thank you#2020-03-0313:35alexmillerCan you back up to what you are trying to do?#2020-03-0317:04geraldodevI'd like to take code action based on the existence of dev alias when connecting to database.#2020-03-0317:05borkdudemaybe use an environment variable for that?#2020-03-0317:13alexmilleryeah, I would add something extra that is explicit for that case, don't rely on all that process stuff above#2020-03-0317:13alexmillersystem property, include dev resources in classpath, etc#2020-03-0318:41dominicmI use system props because they go into jvm-opts and they compose#2020-03-0317:21geraldodevThank you @alexmiller and @borkdude.#2020-03-0603:25devnIs there a good guide or rationale on http://tools.deps vs other options? Several beginners have expressed confusion about lein v http://tools.deps, and lamented the fact indirectly that they’ve now needed to learn both tools while leveling up, but that almost all of their professional work has involved lein.#2020-03-0603:27devnI’m actually confused myself as a long time clojure user. Why did we need another half-finished build tool again? I think I personally understand some of the why, but there’s no doubt it’s contributed confusion to would-be clojure users.#2020-03-0603:33devnThere are numerous answers here that will be acceptable to me, a long-time clojure user, but the fragmentation of beginner tutorials sucks.#2020-03-0603:34devnSorry to be a downer. This ship has sailed, but I have to let y’all know that new users are frequently wondering why they have to learn two build tools.#2020-03-0603:37andy.fingerhutOne answer is that tools.deps is not a build tool. Some more discussion comparing Leiningen vs. tools.deps capabilities here: https://clojureverse.org/t/is-there-a-sales-pitch-for-switching-to-deps-edn-from-lein-in-2020/5367#2020-03-0603:38devnSo, that’s kind of pointing directly at the problem.#2020-03-0603:40andy.fingerhutI can point you at resources that you may or may not find useful to point beginners at. I've got no horse in this race.#2020-03-0603:44porkostomusI think David Nolan made a pretty good case here https://www.youtube.com/watch?v=F0Lv53lop2g#2020-03-0603:45devnI would appreciate any handy resources, and I’ll cop to soapboxing a bit here because it pains me to see beginners who are serious learners being legitimately confused by the ecosystem.#2020-03-0603:47devnI remember when I started, and people were using ivy and gradle and maven. Lein came into fashion, then cake was absorbed into lein.#2020-03-0603:50seancorfield> Why did we need another half-finished build tool again?
CLI/`deps.edn` is very specifically not a build tool.#2020-03-0603:50alexmiller^^#2020-03-0603:51devnyou can explain this all day to new users #2020-03-0603:51seancorfieldNew users do not need a build tool. They need a REPL. They need to run programs.#2020-03-0603:51devnthat’s nonsense imho#2020-03-0603:52devnlike you can be right, but it doesn’t help new user confusion #2020-03-0603:53seancorfieldThe "problem" is that most books and lots of online tutorials that have been around for ages all show Leiningen as part of the first steps in using Clojure. We're in a transition period right now because Leiningen has been around for years and new tooling is appearing. This happens any time a tech community has new tooling come up to replace older, existing tooling.#2020-03-0603:53devnstop right there#2020-03-0603:53devnYou said it’s not a build tool #2020-03-0603:54devnWhat transition?#2020-03-0603:54seancorfieldThere was a time when Boot was gaining ground and folks complained about that too.#2020-03-0603:54seancorfieldLeiningen is a "sledgehammer": it starts a REPL, it runs programs, it bakes, it broils, it takes out the trash.#2020-03-0603:55seancorfieldI see beginners tripping over problems with Leiningen all the time because of plugins they've added to ~/.lein/profiles.clj because it's "easy" rather than "simple".#2020-03-0603:55devnI expect to be on the losing end of this, but I have to really underscore the confusion for new users. I feel it’s my duty. I know that may sound hyperbolic, but this is coming from a place of love.#2020-03-0603:55devnNew users do not understand this subtle distinction you’re making.#2020-03-0603:55porkostomusNot needing Leiningen to get started is a huge plus for new users#2020-03-0603:55seancorfieldA lot of us here are on the front lines in #beginners all the time, helping beginners.#2020-03-0603:56devnyes, and I’ve heard directly from some of them who wonder why anyone is bringing up http://tools.deps, because all of their work is in lein world #2020-03-0603:56devnThat’s exactly the confusion. #2020-03-0603:57seancorfieldThe official Clojure Getting Started docs lead people to the CLI and deps.edn.#2020-03-0603:58devnyes, why? The vast majority of existing production clojure projects don’t use it.#2020-03-0603:58seancorfieldThe latest edition of Programming Clojure uses the CLI and deps.edn I believe (right @alexmiller?) and the ClojureScript Getting Started guide uses the CLI and deps.edn.#2020-03-0603:58seancorfieldTech changes over time. There's nothing strange about that.#2020-03-0603:58seancorfieldLook at all the changes in the Java world over the years... beginners now do stuff completely differently than beginners five, ten, fifteen years ago...#2020-03-0603:59devnI’ve said my piece. I don’t think it’s helpful. I think it’s confusing for new users.#2020-03-0603:59seancorfieldWould you prefer tooling never changed and never improved?#2020-03-0603:59devnthat’s hyperbolic#2020-03-0603:59devnI’m telling you what I know here. Anecdotal as it may be...#2020-03-0604:00seancorfieldI have anecdotes too -- that many beginners find CLI and deps.edn to be much simpler to learn and use.#2020-03-0604:02devnOk, what is depstar, and why does it exist?#2020-03-0604:03seancorfieldAsk the folks who wrote it? 🙂#2020-03-0604:03devnYou maintain a fork of it#2020-03-0604:04devnno?#2020-03-0604:04devnI’m not trump to gotcha anybody#2020-03-0604:04devnBut that project alone should act as sufficient evidence of the confusion #2020-03-0604:04seancorfieldYes, I found the initial version useful enough that I wanted to fix a few bugs in it, and folks seem to like my fork of it.#2020-03-0604:05devnIt uberjars.#2020-03-0604:05devnIt’s. A. Build. Tool.#2020-03-0604:05seancorfieldHardly.#2020-03-0604:05seancorfieldIt's not ant, not maven, nor gradle etc -- those are build tools.#2020-03-0604:06seancorfieldLein and Boot are build tools.#2020-03-0604:06devnagain, this isn’t for me #2020-03-0604:06devnI know what you’re trying to say#2020-03-0604:07devnBut for beginners this is nonsense#2020-03-0604:07devnI’ll leave it there#2020-03-0604:10alexmillerI'm confused what you would find a satisfactory answer to these queries?#2020-03-0604:11seancorfield@devn ^#2020-03-0604:12seancorfield@alexmiller Programming Clojure uses the CLI and deps.edn in the latest edition, right?#2020-03-0604:13alexmilleryes#2020-03-0604:14seancorfieldAnd it doesn't mention "uberjar" anywhere in the book... and only has a couple of mentions of "jar" (I just looked).#2020-03-0604:15seancorfieldSo a beginner can read that book end-to-end and not worry about build tools...#2020-03-0604:15seancorfield(that was what I suspected but wanted to confirm by searching the PDF)#2020-03-0607:40pyrYou could look at it from the angle that there's value in having a lower level toolbox for builds. There is tremendous value as well in having a build tool with conventions (a-la lein). We can hope for a new tool or an evolution of lein, bringing back the baked in conventions which relies on tools-deps#2020-03-0607:52sogaiucheck out the latest insideclojure?#2020-03-0608:15vlaaad“tools.build - shh, top secret”? 🙂#2020-03-0608:50vlaaadA bit late to the flamewar of tools-deps vs build tools, I wanted to repeat once again how much I love clj compared to lein. It all boils down to how you run java programs. To answer the question “How do you run java programs?” one needs to have a look at java command: what arguments it accepts, what are actual, real semantics of running java programs. Actual, real semantics of running java are:
• jvm options like maximum memory (`-Xmx512`) or system properties (`-Dclojure.spec.skip-macros=true`)
• classpath: sequence of directories to look up resources (both .class files and any other type of file that might be needed for running programs) — either as a directories on file system or as archived directories (aka jar files)
• main class to run
• arguments to main class
Everything else is non-essential complexity that is imposed by a build tool.
Some of this added complexity has a lot of value: specifying dependencies to automatically build classpath, for example.
Some of it does not. For example, lets look at source path and resource path separation in leiningen. It’s accidental: you can put your source code in resource files and it will work. What good does it give you besides obscuring actual, real semantics of running java program? Lein plugins, for example, are just wrong and unnecessary abstraction: you could create a separate profile that has a dependency on a tool of your choice and run it. Automatically picked up leiningen profiles for me are still hard to understand: clj’s aliases are much more intuitive and predictable. clj command line options map much better to java command line options. Case in point: yesterday I wanted to run a lein command with system property provided from command line. I could not do it: you have to specify jvm options in :jvm-opts in project.clj, there is no way (at least that I know of) to specify them for lein run invocation.
Beginners coming from java land with equally complected tooling might have a hard time to adapt to tools-deps, but that’s because that tooling equally obscures the actual, real semantics of running java programs.
Break the cycle. Rise above. Focus on simplicity.#2020-03-0608:56dominicmhttps://clojurians.slack.com/archives/C6QH853H8/p1583465670122100
This message was the one I think is quite interesting. The answer is quite easy "there are 2 major competing entrypoints into clojure". I'm sure anybody can understand that. Why is there multiple battery brands that exist, etc. Because they each believe they offer something better than the other.
It's true that there's a cost to having 2 things that both provide the entrypoint. Enough people seem to think that cost pales in comparison to the benefits provided by tdeps. But those benefits are not easily rendered to someone who isn't versed enough in clojure to understand the bigger picture of elements playing out.#2020-03-0609:18teodorluHey @devn,
I think you're bringing some good points. I read your comments above, and I'd like to respond. I hope to be able to bring some civility.
• Personal opinion: getting started with Clojure isn't a perfect experience for new users. That you might encounter project.clj and deps.edn is evidence of that -- learning a single tool would be easier.
• Personal judgement: A common Clojure opinion is that "build tool" is a too broad category, in the sense that Gradle, Maven or NPM works. deps.edn is part of that solution. The "full solution" isn't known yet. deps.edn implementing part of that solution opens the space for other actors.
• Personal experience: A simple / pure data deps.edn is great to work with. I can download the dependencies without having to buy into how you'd like to build your project. Too much complex stuff in the project build pipeline can cause the project build pipeline to become opaque / magic.
Some questions:
1. Do you feel like the above points bring any value to the discussion you'd like?
2. Do you have a suggestion on what work should be prioritized in order to improve new user experience?
Thanks!
Teodor#2020-03-0613:43devnOof, hearing “flame war” and “bring some civility” makes me feel pretty terrible. I talked this out with Alex for awhile last night, and Teodor, I think the second question is one I need to pose to those that have expressed their frustration to me.#2020-03-0615:30devnI sincerely apologize if I came off like a jerk. I was out of sorts last night and did a poor job of broaching the topic and keeping it constructive. facepalm #2020-03-1011:36teodorluYet, I think digging into frustration can be fruitful for improving the state!
I appreciate that you're spending the effort 🙂#2020-03-0615:50seancorfieldIt's definitely unfortunate that Clojure didn't have something like the CLI and deps.edn from "day one" and we ended up with lein as the default : it is easy but not simple in the Clojure sense. At work we started with Leiningen because there really was no choice back in 2011. We switched to Boot in 2015, and the CLI and deps.edn at the end of 2018 I think. It's a much simpler setup, even managing a 90k line monorepo with thirty subprojects. I'm very glad that our tooling continues to evolve. Yes, it means that beginners who start from most books and tutorials today see Leiningen there but then see CLI and deps.edn in new books, new tutorials and on the official site. That's why I consider this a transition : at some point Leiningen will be the minority tooling I suspect or will at least be seen as "the old way" and the question of "which should I use?" will go away, or at least have an obvious answer. #2020-03-0615:52seancorfieldThe transition from monolithic Contrib to new Contrib was similarly confusing to beginners (but the community was much smaller then so the pain was less). #2020-03-0615:53seancorfieldThe introduction of Spec caused similar confusion I think (should I use Schema? Spec? Typed Clojure?).#2020-03-0615:55seancorfieldAnd it's not like we don't have a bunch of options in other areas : which JSON library? Which routing library? Which web server? Learning Clojure is littered with such choices #2020-03-0618:56devnI think I may just fundamentally disagree that simple > easy for new users. I agree there are a lot of choices and that this is just the nature of the beast, but for instance, getting a basic project structure with a working test runner is a whole thing, and IMO it needn’t be this way. test-runner and clj-new exist, but they both are extra steps to simply having a basic working project structure, and then people hit the “ok how do I build an uberjar” step and now they have 3 more options to choose from that I’m aware of. I hadn’t heard about tools.build and am interested in learning more if anyone has a link handy.
lein has its warts certainly. At a certain point you definitely begin fighting with the tool. For instance, checkouts, tasks, the craziness of cljsbuild’s 9000 options, trampoline, passing java opts, and many more, but for new users you could type lein new to get a common project structure, add a dependency, write a test, type lein test, and it just worked.#2020-03-0619:00devnI know this isn’t what http://tools.deps wants to be, but I think introducing it to new users on getting started is a rougher-than-necessary entry point without a bit of ease sprinkled in.#2020-03-0619:06alexmillerI think there are multiple phases. clj is easy, probably easier than lein imo, if you want to just use a repl and write some clojure and use some deps, etc. Beyond that is "make a simple lib/app". Lein is easier in that zone (for now). And then beyond that you get to more bespoke builds, and actually I would again say clj is easier to fully customize because you are not constrained to whatever flags and switches and lifecycles lein makes available.#2020-03-0619:07alexmillermore to come in the near future for the middle zone#2020-03-0619:08alexmillerin the interim, something like sean's template (and others) can give you a slightly opinionated set of choices for making a new project based on clj. It's harder, but it's not like 10x harder, it's like 1.5x harder.#2020-03-0619:25dominicmYou can make easy from simple, can't make simple from easy.#2020-03-0619:29alexmillermore importantly, simple and easy are different dimensions not a vs b.#2020-03-0619:30alexmillerso really, you can go simple+hard -> simple+easy, but you can't go complex+easy -> simple+easy.#2020-03-0619:47devnUnderneath some of what I said last night, there is some personal fatigue from conversations I’ve had where people seem to use simple v easy as a way of downplaying aesthetic or ergonomic concerns by tossing those two words out without addressing the human side. New users should have it easy, full stop. To the detriment of all other things? Of course not. It’s not an either/or.
For example, I have a really difficult time imagining how having clj new and clj test would destroy the aims of the project.#2020-03-0619:47devnI’m not saying this is what should be done necessarily, but there’s no point in being pure to the detriment of new users.#2020-03-0619:58alexmillerI certainly agree that simple does not mean hard#2020-03-0619:59alexmillerclj new and clj test seem like making clj into something that it's not though (a build tool). it's a tool for running programs. template gen and test runners are programs.#2020-03-0620:00devntotally agree#2020-03-0620:00alexmillerhaving ways to discover useful programs to run or templatize/kickstart how to do so seem like good things#2020-03-0620:01devnbut like, should I just make a thing called dmk that does all of the clj stuff and adds two commands for new users?#2020-03-0620:01devnthat seems bizarre to me?#2020-03-0620:02alexmillerwell I wouldn't do that :)#2020-03-0620:02devnhaha, I’m asking then honestly, where would something like that go? It’s never in the cards?#2020-03-0620:03alexmillerI don't have a pat answer to that - finding one is a design process#2020-03-0620:03alexmillerthese things literally exist now#2020-03-0620:03alexmilleras programs that you can run now with clj#2020-03-0620:04alexmillerand as templates that you can create/install#2020-03-0620:05alexmilleris the problem that we have not done a good job explaining that or that that is not easy enough. and if the latter, what would make it so?#2020-03-0620:06alexmillerlike if this page included those examples https://clojure.org/guides/deps_and_cli would that be sufficient?#2020-03-0620:08alexmillerif you added a bit of stuff to your ~/.clojure/deps.edn, then it's literally clj -A:test or clj -A:new etc which seems pretty close to what you have above#2020-03-0620:08vlaaadThere were some discussions on clojureverse to make such a simple tool, even some code was written: https://github.com/Lokeh/plum https://clojureverse.org/t/idea-a-tool-for-helping-developers-get-started-using-clojure-cli-tools-and-deps-edn/4270#2020-03-0620:09devnIt kind of reminds me of clj-rewrite and the evolution of “just update my deps” or “just add the damn test runner”#2020-03-0620:10devnBrb#2020-03-0620:15mike_ananev@devn try this https://github.com/redstarssystems/pbuilder here you can find templates, easy commands and more#2020-03-0620:19mike_ananevin templates I use make utility, so it is very easy to perform different tasks#2020-03-0620:34seancorfieldIf someone does add the :new alias to create a new app or lib project, the generated project includes the :test, :jar/`:uberjar`, :install/`:deploy` aliases so it all becomes as "easy" as lein for basic usage. So the missing piece is really only that initial project creation piece I think.#2020-03-0620:36seancorfieldStrangely, when we switched from lein to boot at work, "new" wasn't a thing for Boot either and I ended up forking lein-new (which wasn't a thing for Leiningen in the very early days, by the way!) into boot-new and it was quite some time before that was integrated into the core tool.#2020-03-0620:36devnHeh, I mean, here we are. http://tools.deps is not a build tool but there are 3 choices for making an uberjar. It’s too simple to add any porcelain commands so there are at least three things that add that type of functionality. That’s my gripe I guess.#2020-03-0620:36seancorfieldSo there's a history of tooling not including an "easy" way to start a new project from scratch 🙂#2020-03-0620:37devnOh yeah! I actually worked on the boot lein template thing I think...#2020-03-0620:37seancorfield> 3 choices for making an uberjar
Oh, c'mon! There are far more than just three choices! :rolling_on_the_floor_laughing:#2020-03-0620:37devnLong time ago#2020-03-0620:37devnhaha#2020-03-0620:38seancorfieldI think far more beginners are stumped by the "how do I build a simple web app" question -- given the plethora of choices they get offered at that point. And heaven help them if someone says lein new luminus myapp and then they have to figure out what that produces!#2020-03-0710:27practicalliI can only share my experiences, but build tools and running a repl are fairly low on the confusion list for those new to coding with Clojure. At ClojureBridge London we have introduced over 300 students who are either new to coding or on their first language. We had been using Leiningen for a long time and now use Clojure CLI and Leiningen,
Clojure CLI is far quicker to install and run, deps.edn has less in it so is easier to explain. As there are numerous plugins (aliases), it's much easier to gradually introduce new features into how Clojure is used. We only really use clj-new and figwheel-main You can also have both Leiningen and deps.edn configuration in a project, so there is a very simple migration (especially if there are relatively few dependencies).
If you have invested a lot of time learning Leiningen or rely heavily on Leiningen plugins that do some magic, like lein-ring, then switching to deps.edn configuration does require a bit of effort, but you don't have to switch.#2020-03-0805:09porkostomusSounds like a great community! I often feel like an oddball for (besides all the usual reasons) choosing to dive into Clojure without significant experience in another language. It has been an interesting ride, and I would love to talk to others who have similar experiences.#2020-03-1011:49teodorluI'd like to add some comments to the discussion on deps.edn's ease of use:
• To me, deps.edn seems like the right platform to make build tools on. It standardizes a part that can and should be standardized.
• How to build a user-friendly product on top of that can then be tacled as the next problem. It would make sense to converge on standard build tooling, but that, to me, seems like the next problem "up the chain".
The Clojure way has seemed to me like "doing the right thing, and letting it take the time it requires". I'm eager to see what build tooling we can have on top of deps.edn when some time has passed.#2020-03-1011:51teodorluRelevant dichotomy: bottom up/ top down. Bottom-up: small functions, composition. Platform first. Common with Clojurians. Top-down: focus foremost on the end-user experience, and try to improve that incrementally. Common in JS-land.
To me, bottom-up rhymes with simple-first, whereas top-down rhymes with easy-first.#2020-03-1011:51teodorluRelevant dichotomy: bottom up/ top down. Bottom-up: small functions, composition. Platform first. Common with Clojurians. Top-down: focus foremost on the end-user experience, and try to improve that incrementally. Common in JS-land.
To me, bottom-up rhymes with simple-first, whereas top-down rhymes with easy-first.#2020-03-1017:59rickmoynihanInteresting observations. I guess neither top-down nor bottom up guarantee either simplicity or ease in the final result; but yeah I think what you’re saying generally holds. And you can also do both together and meet in the middle. As a heuristic though if you wanted simplicity you’d tend to start bottom up, and if you wanted ease you might prefer top-down — though they’re not necessarily mutually exclusive.#2020-03-1109:09slipsetOne of the things that I find interesting in this perspective is that Clojure has slurp/spit whereas I’ve never figured out how to read a file in Java (prior to Java7)
Here it seems like Clojure has taken the top down approach, whereas Java took the bottom up approach with the streams and readers and what not.#2020-03-1109:21rickmoynihanNah, clojure also has the simple approach too with slurp makes the common case easy.#2020-03-1120:09unbalancedno sure if this is a tool-deps specific question exactly, does anyone have any suggestions on how to troubleshoot this?
$ clj -R:nREPL figwheel.main -bo min
Cloning:
#2020-03-1120:10unbalancedI am able to clone it directly with git#2020-03-1120:10alexmillerwhat os?#2020-03-1120:10unbalancedred hat >.<#2020-03-1120:11unbalanced$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL=""
BUG_REPORT_URL=""
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
#2020-03-1120:11alexmillerit uses the ssh agent to obtain identities#2020-03-1120:11alexmillerso - is that running?#2020-03-1120:11alexmillerssh-agent#2020-03-1120:11alexmillerif so, does it have your identity ssh-add -l#2020-03-1120:11alexmillerif not, add it#2020-03-1120:12unbalancedchecking#2020-03-1120:12alexmillerthose cover 90% of these issues#2020-03-1120:13alexmillerhttps://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent is a good overview of those steps too#2020-03-1120:14unbalancedappreciated. I turned it on and got a new error. Trying to figure out how to add my key now.#2020-03-1120:15alexmillerssh-add ~/.ssh/id_rsa
#2020-03-1120:15unbalanced$ clj -R:nREPL figwheel.main -bo min
Cloning:
#2020-03-1120:16unbalanceddoes that seem like a different error?#2020-03-1120:16alexmilleryeah, that seems bad#2020-03-1120:16unbalanced(after following steps of turning on ssh-agent and adding key)#2020-03-1120:16unbalancedhahahaha#2020-03-1120:16alexmillermight be in how you're running ssh-agent#2020-03-1120:16unbalancedI believe you#2020-03-1120:16alexmilleryou've walked off my knowledge in this area :)#2020-03-1120:17unbalancedalright! good start. much appreciated#2020-03-1120:19alexmillersudo mount /tmp -o remount,exec internet suggests maybe /tmp needs exec ?#2020-03-1120:20unbalancedhmm#2020-03-1120:21alexmilleror set a different tmp dir#2020-03-1120:22alexmillerclj -.tmpdir=/my/tmp -R:nREPL figwheel.main -bo min#2020-03-1120:23alexmillerthe git ssh stuff happens through jsch which talks to ssh-agent via jna. it appears jna will make and use stuff in the tmp dir#2020-03-1120:24alexmillerdifferent lib, same problem: https://support.cloudbees.com/hc/en-us/articles/215281717-Jenkins-fails-to-start-with-JNA-error and pretty good writeup of the alternatives#2020-03-1120:24unbalancedinteresting#2020-03-1120:24alexmillerI've never seen this before, so you get a cookie#2020-03-1120:24alexmiller🍪#2020-03-1120:24unbalancedhahahahahaha#2020-03-1120:24unbalancedyesssss#2020-03-1120:26unbalancedalmost done setting all this up jeeze#2020-03-1120:27unbalancednow I'm getting UnknownHost key, but at least we're making progress in some direction#2020-03-1120:28unbalancedof course I see there are 6 ssh-agents running now 😑#2020-03-1120:33alexmillerwell you might end up running into https://clojure.atlassian.net/browse/TDEPS-104#2020-03-1120:35unbalancedhttps://stackoverflow.com/questions/2003419/com-jcraft-jsch-jschexception-unknownhostkey#2020-03-1120:35unbalancedthat fixed it#2020-03-1120:35alexmiller🍪#2020-03-1120:36unbalanced😮 holy snap#2020-03-1120:37unbalancedthat's goin on the fridge#2020-03-1120:37unbalancedokay last problem, most of the things downloaded#2020-03-1120:38unbalancedError building classpath. Failed to read artifact descriptor for org.clojure:clojure:jar:1.10.1
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:clojure:jar:1.10.1
#2020-03-1120:38unbalanced@alexmiller is there a way to comment on that issue?#2020-03-1120:39unbalancedor just... include that link in there I guess#2020-03-1120:39unbalancedspecifically this is what I did:
$ ssh-keyscan -H -t rsa >> known_hosts
#2020-03-1120:42alexmillerdone#2020-03-1120:50unbalancedlast piece of the puzzle was a stale .m2 repo that had wrong perms#2020-03-1120:51unbalanceddo you know where the git stuff gets cloned to? I don't see it in .m2#2020-03-1120:56unbalanced.gitlibs#2020-03-1120:56unbalancedsometimes it helps if I just post the questions and then get embarrassed and try harder#2020-03-1121:31alexmillersome of these questions were answerable at https://clojure.org/reference/deps_and_cli btw#2020-03-1216:04danielcomptonI'm wanting to write a small Java file and use it in my Clojure app using deps.edn and clj . Is there anything similar to :java-source-paths or another way I can accomplish the same goal? My other option is to write this and create a JAR for it and deploy it somewhere or keep it as a local resource#2020-03-1216:05dominicmNo equivalent. No compilation done by tdeps.#2020-03-1216:06dominicmYou can use javac and have it output to target, and run that shell script before starting clj#2020-03-1216:18danielcomptonMight be easier to rewrite the algorithm to Clojure 🙂#2020-03-1216:39alexmillerthere are some tools.deps add-ons that can compile as well, like badigeon#2020-03-1216:48mike_ananev@danielcompton try this https://github.com/redstarssystems/pbuilder#2020-03-1216:48mike_ananevit's a wrapper for badigeon#2020-03-1216:49borkdudemaybe a silly question, but why does badigeon need a wrapper? it's already a tool?#2020-03-1216:49mike_ananevalso there is a template for application https://github.com/redstarssystems/app-template#2020-03-1216:50mike_ananev@borkdude no, badigeon is not a ready tool. it is low level api for managing project distributed as sources, which need to be adapted.#2020-03-1216:51borkdudeah ok, I see now#2020-03-1217:52dominicmJavac is pretty good though#2020-03-1220:48alexmiller@dominicm if you are suggesting that libraries should maintain a directory full of compiled classes, I think that is a bad idea and you should not do that#2020-03-1220:51alexmillerif you want to distribute compiled artifacts, you can already do that by putting them in a jar. we don't typically do that in Clojure because the source is so much more flexible.#2020-03-1220:53dominicmI'm not. I'm saying all of the infrastructure around this, code laying around to do the compilation on startup.#2020-03-1220:53imre@alexmiller if you call (compile 'foo) twice, will the second be almost a no-op as it sees it's already compiled?#2020-03-1220:57alexmilleryes#2020-03-1220:58alexmilleryou can try it in that project I posted if you like#2020-03-1221:03imrecheers#2020-03-1316:35unbalancedI'm back with more git issues#2020-03-1316:36unbalancedCloning:
#2020-03-1316:36unbalancedtrying to help a coworker get it running#2020-03-1316:36unbalancedthis is on mac#2020-03-1316:37unbalancedso far we've done the following#2020-03-1316:37unbalancedif [ -z "$SSH_AUTH_SOCK" ] ; then
eval `ssh-agent -s`
ssh-add
ssh-add -k ~/.ssh/id_rsa # add keys -- did this for all private keys
# add github keys
ssh-keyscan -H -t rsa >> ~/.ssh/known_hosts
fi
#2020-03-1316:39unbalancedgit clone works#2020-03-1316:40ghadido they also have ~/.ssh/config entries? @goomba#2020-03-1316:41unbalancedI haven't checked -- is that important?#2020-03-1316:42unbalancedthere is no ssh config#2020-03-1316:42unbalanced#2020-03-1316:43unbalancedwow#2020-03-1316:43unbalancedrenamed stuff and it worked#2020-03-1316:44unbalancedhow on earth did you know that#2020-03-1316:47alexmillerwhat error are you getting?#2020-03-1316:47unbalancedorg.eclipse.jgit.api.errors.TransportException: #2020-03-1316:47unbalancedit was tied to the .ssh/config somehow#2020-03-1316:51unbalanced(it's working now, thank you @ghadi)#2020-03-1316:51ghadiJSch, the underlying library used, tries to read the .ssh/config for some things instead of consulting the running ssh-agent#2020-03-1316:53unbalancedI notice that https://github.com/clojure/tools.gitlibs says it supports:
• Full sha (40 chars)
• Prefix sha (sufficiently unique in the repo)
• Tag name
• Branch name
#2020-03-1316:53unbalancedI don't suppose :git/url supports tags/branches, and/or would a PR to that effect be welcome?#2020-03-1316:54unbalancedI'm using this extensively and I have 11 (and growing) repos (we're using enterprise git so I can't publish to clojars) and being able to tags/branches would be nice#2020-03-1316:55alexmillerso as above tools.gitlibs does support tags or branches#2020-03-1316:55alexmillerclj intentionally supports only full shas#2020-03-1316:56unbalancedbecause tags/branches are mutable ?#2020-03-1316:56alexmilleryes, it totally breaks classpath caching#2020-03-1316:56unbalancedtags too?#2020-03-1316:56alexmillersame#2020-03-1317:13dominicmWell, tags are meant to be immutable. You have to force push them.#2020-03-1317:27alexmilleryou see how somebody else could break your build with that though, right?#2020-03-1317:45dominicmOh yeah. I wasn't sure how defensive we were being really. I understand that one changes future builds, and one is cryptographically difficult to change future builds. If you want cryptographic protection then it's okay.#2020-03-1318:06dominicmMaven central do that through policy, but there's no requirement that happens though (I think)#2020-03-1316:57unbalancedgotcha... bummer#2020-03-1316:57unbalancedI guess including a "force recalculate" directive would be out of the question?#2020-03-1316:57alexmillerthat does exist (-Sforce) but you would need to basically do it on every clj call#2020-03-1316:58alexmillerI would not rule out some way to support this use case, but would want to start from the big picture#2020-03-1316:59unbalancedgotcha. I'm just trying to make the workflow for this work in a less error-prone way.#2020-03-1316:59alexmillerthe other similar but different thing people have been poking at is monorepo with embedded local deps#2020-03-1317:00unbalancedI'm gonna DM you the details that I can't put here on what the situation is#2020-03-1317:01alexmilleranother you might want to consider is using a private shared repo (either something like nexus or even a private s3 repo)#2020-03-1317:02unbalancedso for each of those repos you see ... every time I make a commit, I need to update the shas. And then I need to update the READMEs for each one. And god forbid they have dependencies on one another --- need to do all that by hand#2020-03-1317:02alexmillerI get it - I've used a private shared repo in this situation#2020-03-1317:02unbalancedis that like a "private clojars"?#2020-03-1317:02alexmilleryeah#2020-03-1317:02unbalancedok -- that would be a much better solution, yes#2020-03-1317:02alexmillermaven repos can be just a file system#2020-03-1317:03alexmillerand private s3 repos are exactly that#2020-03-1317:03unbalancedI did not know that. How do you specify an alternate .m2?#2020-03-1317:03alexmillernexus is actually an app and can do a lot more for you (like mirroring all your external deps in case maven central is unavailable or something)#2020-03-1317:03alexmilleryou don't need an alternate .m2#2020-03-1317:04alexmillerfor this#2020-03-1317:04alexmillerif you're asking separately, you can use a :mvn/local-repo key in your deps.edn#2020-03-1317:04unbalancedinnnteresting#2020-03-1317:04alexmillerhttps://clojure.org/reference/deps_and_cli has docs on private s3 access#2020-03-1317:04unbalancedokay, thank you, I'll look into that. 🙏#2020-03-1317:05alexmillerfor deployment, you'll need something to make artifacts and deploy (really just an s3 copy) and there are some clj tools out there https://github.com/clojure/tools.deps.alpha/wiki/Tools#2020-03-1317:08unbalancedwhich category on tools would that be under?#2020-03-1317:09unbalancedversioning and releasing?#2020-03-1317:10alexmillerwhich part?#2020-03-1317:10alexmillermaking an artifact -> packaging#2020-03-1317:11alexmillerbut I guess also some of versioning and releasing#2020-03-1317:11unbalancedno I meant a tool for where to host#2020-03-1317:11unbalancedthe files#2020-03-1317:11unbalancedi.e. nexus or similar#2020-03-1317:12alexmilleroh, I don't think you'll find that theree#2020-03-1317:12unbalancedjust making sure I'm not crazy#2020-03-1317:12alexmillerbut you could look at any maven repository docs. https://www.deps.co/ is built by Daniel Compton from the Clojure community#2020-03-1317:13unbalanced😛 well I am technically a little @goomba#2020-03-1317:13alexmillerhttps://www.sonatype.com/product-nexus-repository#2020-03-1317:13alexmillerhttps://jfrog.com/artifactory/#2020-03-1317:13alexmillerthose are common ones#2020-03-1317:14alexmillerI think there's a thing built into git too?#2020-03-1317:15unbalancedWHAAAT#2020-03-1317:15unbalancedyay that's a tool I'm actually allowed to use!#2020-03-1317:15alexmillerhttps://dzone.com/articles/using-github-as-maven-repository#2020-03-1317:15alexmillerjust googling, not sure that particular article is the platonic ideal#2020-03-1317:17alexmillerthere's some tool for this too that I can't remember the name of#2020-03-1317:18alexmillerhttps://jitpack.io/#2020-03-1317:40dharriganI use Nexus OSS#2020-03-1317:40dharriganworks well#2020-03-1318:01alexmillerditto#2020-03-1319:28lilactowna user reported an error trying to use git deps for a lib I wrote:
https://github.com/Lokeh/helix/issues/5#issuecomment-598850716
does anyone know offhand, based on the message, whether this is a user error or something with my lib?#2020-03-1319:32borkdudemaybe he can try -Sforce?#2020-03-1320:00hiredmanthey should delete .gitlibs/libs/helix#2020-03-1320:12hiredmanor maybe .gitlibs/libs/Lokeh/helix#2020-03-1320:15hiredmanit is a jgit error you get when trying to clone into a none empty directory https://github.com/eclipse/jgit/blob/cf60f26d26db2dc4f2d46264d5417e51c4e05c91/org.eclipse.jgit/resources/org/eclipse/jgit/internal/JGitText.properties#L126#2020-03-1515:23teodorluI'd like to comment on the previous discussion of referencing with tags.
• Personal opinion: a utility to rewrite a git ref (HEAD, some-tag, some SHA letters) into a full-blown 40-char SHA might help ease of use. That may be a task for a more high-level tool, though; it seems like more of a UI concern. Perhaps an alias.
• Personal opinions: refs should be resolved immediately, and only SHAs make it into the data
clj -A:install-dep --git-url --git-ref HEAD
I'm guessing this might not be novel ideas, as I feel that tools.deps already embodies this kind of thinking.#2020-03-1516:24alexmillerHave you looked at -Sresolve-tags option#2020-03-1516:25alexmillerIt resolves tags to shas in your deps.edn#2020-03-1618:43teodorluYou're way ahead of me 💯#2020-03-1523:19Jag GunawardanaI want to include my database migrations in my Uberjar. I am using juxt edge with onejar to build. In lein resource-paths did this. I am using ragtime with edn files. The only way I can get this to work is using “jar uf...” to update the jar after building it. There must be some deps .edn magic?#2020-03-1523:41seancorfield@jag I've no idea how Juxt/OneJar handle that but with depstar you would just use an alias that adds the resources (via :extra-paths) when you ask it to build the JAR.#2020-03-1523:42seancorfieldWe build uberjars at work using depstar that include DB migrations, HTML templates, and all sorts of non-code stuff.#2020-03-1608:20dominicm@jag they should be included by default unless they're not in a default alias. If they aren't you can put "-A:alias" to pack.#2020-03-1615:59Jag GunawardanaThanks for the suggestions, looks like moving to depstar seems to be the best long term solution. I like the approach taken in juxt edge project, but the docs around build are a bit too concise.#2020-03-1616:40dominicmSorry about that, is there somewhere I can improve them?#2020-03-1808:53Jag GunawardanaI think a few more use cases for some of the tools, or general cases would work e.g. “How to include resources in the JAR file”. I ended up using Sean’s fork of depstar with edge.
clojure -A:prod:depstar -m hf.depstar.uberjar myjar.jar
and running
java -cp myjar.jar clojure.main -m edge.main
That seems to work fine.
I can add something to the docs on this if it helps?#2020-03-1810:29dominicmI'd love to take a look at a pr if you have an idea for this :)#2020-03-1915:37Eamonn SullivanHi, probably a quick question: Can I use a private github repo as a dependency in deps.edn? I'm trying, but getting Authentication is required but no CredentialsProvider has been registered. If it is supported, how do I provide my credentials (a github access token, I guess)?#2020-03-1915:39ghadiyou can, but through ssh:// + an ssh-agent, not via the https:// url#2020-03-1915:39ghadiyou can, but through ssh:// + an ssh-agent, not via the https:// url#2020-03-1915:45Eamonn SullivanI've been trying /cdn-cgi/l/email-protection links, which work from the command-line (`git clone /cdn-cgi/l/email-protection`), but when I try that in a deps.edn I get invalid privatekey. I must be missing something simple...#2020-03-1915:50Eamonn SullivanSorry, I thought I hadn't run ssh-agent and it was working, but I was actually in the wrong directory. 😞#2020-03-1915:53ghadiplease describe exactly what you did, and what the result was#2020-03-1915:57Eamonn SullivanI added to the dependencies. My private repo is the last one listed below. I then try to run clj and the result is Error building classpath. #2020-03-1916:01ghadiwhat is the output of ssh-add -l#2020-03-1916:01ghadiand is your key an ed25519 key?#2020-03-1916:02Eamonn Sullivanhere it is.#2020-03-1916:03Eamonn SullivanAnd, no, it's not an ed25519 key, as far as I can tell (they both say RSA).#2020-03-1916:06ghadiif you have a bunch of entries like defaults in $HOME/.ssh/config consider moving that file out of the way and retrying#2020-03-1916:07ghaditemporarily#2020-03-1916:07ghadithe ssh library prioritizes that file even when there is an ssh-agent present#2020-03-1916:09ghadi(unfortunately)#2020-03-1916:09Eamonn SullivanAh, that worked. I need those entries in config for various things, so I wonder what I need to do to not confuse things.#2020-03-1916:12Eamonn SullivanBut, anyway, I can now build, so that's a big improvement. Thank you!#2020-03-1916:13ghadimy pleasure#2020-03-1921:39samI’m getting an unexpected transitive dependency resolution in my deps.edn project. This is what my dependency tree looks like:
my-project -> direct-dep -> trans-dep1 (version 1)
\
-------> trans-dep2 -> trans-dep1 (version 2)
• direct-dep is a git dependency
• direct-dep has a pom.xml file that serves as the manifest
The problem is that trans-dep1 (version 2) is on the classpath instead of version 1#2020-03-1921:41samAm I correct in thinking that version 1 of trans-dep1 should be the expected resolution (assuming I’m describing this correctly and am not doing something dumb I’ve overlooked)#2020-03-1921:43ghaditools.deps prefers latest versions first#2020-03-1921:52samOh thanks, I totally missed that.#2020-03-1921:53samIs there a rationale for that written somewhere?#2020-03-1921:56alexmillerthe long form is Rich's "Spec-ulation" talk https://www.youtube.com/watch?v=oyLBGkS5ICk#2020-03-1921:57alexmillerand I've done a couple talks that cover it as well, most recently (https://www.youtube.com/watch?v=7CM7Ef-dPWQ)#2020-03-1921:57samAwesome, thanks. I appreciate it.#2020-03-1921:58alexmillerI don't know if I have a good written thing to point at#2020-03-1922:06samNp, I think I’ve seen the spec-ulation talk. I think I vaguely remember this. In any case, I can imagine an argument that chosing the latest version is simpler, easier-to-understand behavior compared to the behavior in lein/mvn that I was expecting.#2020-03-1922:25seancorfieldThe latest episode of the ClojureScript Podcast is very good https://clojurescriptpodcast.com/ -- Alex talks about tools.deps and covers this specific issue...#2020-03-1922:30alexmilleroh yeah, that too. we recorded it months ago so it's hard to remember that. :)#2020-03-1922:40seancorfieldIt's a really good all-round discussion about tools.deps and the new CLI stuff. Definitely worth pointing folks at for any Qs regarding the "philosophy" of it (including the SemVer stuff).#2020-03-2015:38grzmI have a project which has a deps.edn dependency to a private https://github.com repo, Clojure deps is failing to pull the dependency with:
Cloning:
Seems like it’s an issue with my SSH keys, but I’m a bit puzzled as I can clone this same dependency from https://github.com from the command line with git clone without any problem, and the necessary keys show up when using ssh-add -l. Hints on what I might be doing wrong?#2020-03-2015:40Derek Passenhttps://clojurians.slack.com/archives/C6QH853H8/p1584634015008000#2020-03-2023:53colinkahnIs this the best guide to use for getting running on Windows? https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#2020-03-2109:16practicallithe #clj-on-windows channel can also provide help with this#2020-03-2201:22colinkahnOh nice, thanks!#2020-03-2023:57sogaiuit definitely has relevant info that people have referenced here from time to time.
the scoop option mentioned at the bottom of the page appears to be somewhat popular. would recommend checking that out.#2020-03-2023:58sogaiui have been using that successfully.#2020-03-2123:29practicalliHow would I include the Java sources in the :path of my deps.edn configuration?
The sources are currently in a src.zip file, from the install. Is it as simple as using "src.zip" to the :path configuration? Or would I need to extract and a the specific directories relative to the Clojure project?
Also wondering how to include these sources to multiple projects, either using an alias in ~/.clojure/deps.edn or creating a symbolic link to the src.zip file (or extracted directory).#2020-03-2123:49andy.fingerhutNeither the clj nor clojure commands will do anything with Java sources, unless you somehow use them to run a program that looks for and does something with Java source files.#2020-03-2123:51andy.fingerhute.g. if you are using something like mevyn program to compile Java source files: https://github.com/clojure/tools.deps.alpha/wiki/Tools#2020-03-2201:07potetmOR you can compile the src w/ javac into target/classes and add "target/classes" to your :paths#2020-03-2206:40mike_ananev@U05254DQM try https://github.com/redstarssystems/pbuilder#2020-03-2207:34dominicmI think the src.zip is referring to the ones that come with the jdk. They're used by cider to allow you to jump to source on java code.#2020-03-2207:35dominicm@U05254DQM I think zip files are fine on the classpath. A :local/root should be fine.#2020-03-2208:41practicalli@U09LZR36F that sounds what I am looking for, thank you.#2020-03-2211:15practicalliIt works. I have a deps.edn as follows
{:path
["src" "resources"]
:deps
{org.clojure/clojure {:mvn/version "1.10.1"}
;; Java 11 sources
java-sources {:local/root "/usr/lib/jvm/openjdk-11/lib/src.zip"}}}
I can now browse java sources in Spacemacs with clj-find-var . Thanks everyone.#2020-03-2213:09dominicmRemember that the location will vary for everyone :)#2020-03-2211:14borkdudeEnjoyed the podcast Alex, thanks for taking the time! https://clojurescriptpodcast.com/#2020-03-2211:19pithylessTrying to debug this kind of warning:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:~/.m2/repository/org/slf4j/slf4j-nop/1.7.25/slf4j-nop-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:~/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
When I run clojure -Stree I do not see any mention of slf4j-nop. Any ideas on how I can find out what is pulling in this dependency (so I can add an :exclusion)?#2020-03-2211:26pithyless^ Nevermind, PEBCAK.#2020-03-2414:07holyjakHi! I have added :mvn/repos {"my-company" {:url "...."}} but I want deps to also resolve dependencies from my maven local repo. How do I add it, preferably w/o hardcoding the path of my home directory?#2020-03-2414:08alexmillerit will already do that#2020-03-2414:08alexmillerassuming you mean ~/.m2/repository - that's actually not a repo, but a local cache of external repos#2020-03-2414:10alexmilleras a cache, it's checked first though so you can install stuff locally that you never deploy and it will be used#2020-03-2414:15holyjakthanks, that is exactly my use case. I found out that the jar was wrongly published to the local cache.#2020-03-2423:29athomasoriginalTerminology is always as touchy topic, but would it be fair to refer as tools.deps.alpha as a package manager? If not, how would you describe the distinction?
The goal for myself to group clojure tools into buckets that fit my mental model. I am happy to update my model, but I just want to see if my current understanding fits 🙂#2020-03-2423:36andy.fingerhutI do not know of any precise definition of the term "package manager" that everyone would agree with, across different programming languages, but it seems to me that tools that are called by that name often do more things than tools.deps.alpha does.#2020-03-2423:36andy.fingerhutThis is too many words, but the http://clojure.org page introducing tools.deps.alpha describes it thus: "a library providing a functional API for resolving dependency graphs and building classpaths that can utilize both Maven and other providers of code or artifacts"#2020-03-2423:37andy.fingerhutPerhaps calling it a "dependency resolver" briefly captures the essentials?#2020-03-2423:37andy.fingerhuthttps://clojure.org/reference/deps_and_cli#2020-03-2423:40athomasoriginalMy thoughts exactly, Andy. I agree that it would be a stretch to compare it as this would lead to further comparisons to other tools like npm#2020-03-2501:44alexmillertools.deps is fundamentally about making classpaths, and incidentally about transitive dependency resolution#2020-03-2501:45alexmillerI would not call it a package manager per se#2020-03-2501:45alexmillerclj is about running programs#2020-03-2510:21rickmoynihanI’ve been meaning to speak about this issue for a while; not sure if others have run into it or not.
I have a private library in a private github repo; that we’re requiring via tools.deps.
Every so often, in circumstances I don’t fully understand, but it usually appears to be when I do things like change branches (and I suspect the sha in that dep has changed) I get the following exception:#2020-03-2510:21rickmoynihan#2020-03-2510:25rickmoynihanWhen it occurs I usually spot that my ssh-agent is somehow no longer running; so restart that but then get:#2020-03-2510:25rickmoynihan#2020-03-2510:27rickmoynihanThen clearing the lock file fixes it#2020-03-2512:19alexmillerWe’ve had people report this but I’ve never been able to repro it#2020-03-2514:00rickmoynihanIt happens to me quite often#2020-03-2514:05rickmoynihanthere are a few other things that might be relevant:
1. some-private-lib is actually included in our deps.edn multiple times; with different :deps/root s
2. In dev I start a bunch of services with overmind a Procfile process manager. It boots them in parallel. One of those services is my clj repl; another is shadow-cljs that references the same deps.edn; so when resolving deps two processes are occaisionally concurrently resolving/fetching deps in parallel.#2020-03-2514:07rickmoynihanI suspect 2 might be problematic… so sometimes try and resolve the deps first; then run overmind… though I do at times forget — it usually seems to work without any apparent problems, though it does seem a bit dangerous on my part.#2020-03-2514:09Eamonn SullivanI saw the first error just the other day. I had two small projects, one using a github ref using one hash (that checked out the repo fine) and then moved to another with a newer hash (but the same repo) and saw the above error when trying to build. I went into the directory and deleted the repo and tried again and it worked. Is that supposed to work?#2020-03-2514:11rickmoynihanI think I’ve seen it a similar scenario to that too#2020-03-2514:11rickmoynihanbut this one is definitely within a single project — though changing branches; with a changing sha on the dep which probably amounts to the same thing#2020-03-2514:16alexmillerwith the newer clj's, resolution happens in parallel, so even in a single project, you could encounter that#2020-03-2514:19alexmilleryou can turn that off with -Sthreads 1#2020-03-2514:42rickmoynihanOk but this is in a single project; but with two concurrent processes potentially resolving the same deps with tools.deps. Is that safe if the processes aren’t coordinated?#2020-03-2515:12alexmillerdon't know#2020-03-2512:20alexmillerThe first error that is#2020-03-2513:36WhittlesJrHi,
I love the ability of tools-deps to target git repos at a certain hash. I've operated under the assumption that the hostname of the repo will stay the same forever for private repo dependencies. But that's proven not always to be the case.
If a repo moves to a new hostname, it breaks every historical project state that referenced it.
Yocto has a feature called a "repo manifest" that allows the repository configuration to be a lot more fluid, and effectively solves this problem. Does tools-deps have an equivalent option?#2020-03-2513:38alexmillerno#2020-03-2513:39alexmillerwhat is the repo manifest?#2020-03-2513:39alexmillergot a link or something?#2020-03-2513:47WhittlesJrI think this is what I'm talking about:
https://www.yoctoproject.org/docs/2.7/bitbake-user-manual/bitbake-user-manual.html#repo-fetcher
And an example:
https://github.com/texierp/yocto-bsp-manifest/blob/master/default.xml#2020-03-2513:49WhittlesJrI suppose there are workarounds for the issue at the OS-level, but this would be a pretty slick feature.#2020-03-2514:05dominicm@alex.joseph.whitt is this like having a proxy or alternative source for a repo? I'm struggling to understand.#2020-03-2514:16WhittlesJrI'm not a Yocto user myself, but I believe the idea is that you can have a locally-defined manifest of repo sources. For each dependency, Yocto will look through them all in order and try to resolve the dependency in each.
That solves the problem of trying to build an old version of your project after your git server hostname changes. The problem is really only sticky when you have multiple internal Clojure dependencies that all point to the same hostname. You have to go through each dependency, branch off of the contemporary commit, update the hostname, commit, then update the hashes on projects that pull in that dependency. Kind of hellish unless you redirect the old hostname at the DNS level or something.#2020-03-2514:21alexmillerI mean, you are defining a lib and a coordinate (with git config) in your deps.edn. You can change that coordinate/git config but keep the lib name to point it elsewhere.#2020-03-2514:26alexmillerI guess there could be a layer of indirection added in front of the git url#2020-03-2514:26alexmillerneeds a lot more thought#2020-03-2514:51dominicmAnother use case for this: the bank I work at has a github mirror which is both faster and allows clones from machines that otherwise can't.
This would be useful for us.#2020-03-2610:31daveliepmannHi folks, I've got a head-scratcher with transitive dependencies. I found a workaround but it doesn't seem right.
We have a new library which depends on a Java lib living in the BeDataDriven public repository: http://docs.renjin.org/en/latest/library/project-setup.html
We wish to consume this library in a separate project. However, we get an error when starting a REPL via clojure CLI (or CIDER):
Error building classpath. Could not find artifact org.renjin:renjin-script-engine:jar:3.5-beta43 in central ()
This error goes away if I include the BeDataDriven repo to the consumer's :mvn/repos. Like I said, this is a fix but it doesn't feel correct. Is this by design?
I've created a minimal reproduction (three tiny files) of the issue in this repo: https://github.com/daveliepmann/minimal-transitive-deps-example The workaround is commented out. I get the error above when running clojure from the command line in min-consumer directory. I am running the latest clojure from brew.
The actual lib is at https://github.com/appliedsciencestudio/rdata/ and the consumer deps.edn fails with both git and local deps:
clojure
appliedsciencestudio/rdata ;; {:local/root "/Users/daveliepmann/src/forks/rdata"}
{:git/url ""
:sha "6dff85aa7b83a9ce85eb8d70b0672a3adf0dc7f2"}
Prior art: this issue looks like it could be related to https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-127?filter=allissues&orderby=priority%20DESC&keyword=transitive but that seems to have been fixed.#2020-03-2610:45dominicmI don't think repos are conveyed across deps.edn projects. They're considered local like :aliases#2020-03-2610:47daveliepmannRight, but the consumer lib doesn't know, or in my mind shouldn't care, that the lib it's requiring uses a repo that's not Maven or Clojars.#2020-03-2610:47dominicmhttps://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-46?filter=allissues&orderby=priority%20DESC&keyword=transitive @daveliepmann#2020-03-2610:48daveliepmannYup that looks like it, thanks. I missed it in my Jira search.#2020-03-2610:48dominicmI can think of a couple cases where that's not true. Banks being one, repos are not transparent there. There's also the risk of someone adding a repo that them causes shadowing. I'm not sure how maven works for this.#2020-03-2611:45rickmoynihanI might be misunderstanding the problem here - but isn’t it a security risk for a transitive dep to specify a new :mvn/repo; if we’re not careful might it not cause tools deps to fetch other deps from there too?
Granted 3rd party deps are always a security problem; but this suggested change in behaviour seems like it would obfuscate things further and potentially makes auditing harder?
I think the usecase is interesting; but would it not be safer for libs containing :mvn/repos to cause tools.deps to fail with a message saying “add X to :mvn/repos”?#2020-03-2611:49rickmoynihanAlso couldn’t it open up new attack vectors? e.g. an attacker could possibly set it to a private repo you own; and cause you to inadvertantly package secret deps in published binaries etc? Kind of like a CSRF attack.
Admitedly this is a fanciful attack — and it would require you to depend on an attackers library either explicitly or transitively -- but attacks usually are elaborate.#2020-03-2611:55dominicmThat's right. But maven allows this, so it might already be the expected model.#2020-03-2612:22alexmillerYes, this should work, just not handled right now#2020-03-2613:00ghadiThat attack vector sounds realistic#2020-03-2613:09alexmillerit needs some thought. imo, this is an under-considered part of the Maven model in general. repos are identified by name, and those names are just totally arbitrary and unique solely by rarity/convention.#2020-03-2613:10alexmillerbut Maven does certainly let you add repos in transitive deps for this case. I'm not sure if their use is "scoped" to that dep and below though#2020-03-2613:46rickmoynihanyeah was wondering about this too… seems less risky to scope down the subtree; than apply globally.#2020-03-2613:12alexmillerthey are somewhat shielded from this by using a top-down, breadth-first, first one found wins strategy as "parents" in the tree have already been resolved before the transitive repo is encountered#2020-03-2613:16alexmillerthat's kind of also true with tools.deps (resolution happens top-down), but selection can be modified during the traversal and the "source" is not tracked right now in any way#2020-03-2703:39dorabUsing Sean Corfield's fork of depstar, and running the command clj -A:uberjar x.jar, I get
Building uber jar: x.jar
{:warning "clashing jar item", :path "META-INF/services/com.fasterxml.jackson.core.JsonFactory", :strategy :concat-lines}
{:warning "clashing jar item", :path "META-INF/services/com.fasterxml.jackson.core.JsonFactory", :strategy :concat-lines}
I know it is just a warning, but what do I do to fix it? My deps.edn is
{:paths ["src"] ;until TDEPS-52 is fixed
:aliases
{:test
{:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "RELEASE"}}}
:runner
{:extra-deps
{com.cognitect/test-runner
{:git/url ""
:sha "76568540e7f40268ad2b646110f237a60295fa3c"}}
:main-opts ["-m" "cognitect.test-runner" "-d" "test"]}}
:deps
{
cheshire {:mvn/version "5.10.0"}
http-kit {:mvn/version "2.3.0"}
org.clojure/clojure {:mvn/version "1.10.1"}
}
}
If this is not the appropriate channel, please redirect me. Thanks.#2020-03-2703:49seancorfieldYou don't do anything to fix it. It's just warning you, so you know that the same file is being copied into your JAR multiple times (and it is handling it by concatenating the files). Jackson is a horribly problematic library -- it is the cause of most of the version conflicts we see people run into these days but unfortunately it's also fairly ubiquitous so...#2020-03-2703:50seancorfieldI'm a bit surprised you're seeing the message twice -- that implies three copies of the file are being found on the classpaths...#2020-03-2703:52seancorfieldThe 1.0.x version of depstar (currently on master) will have a -S / --suppress-clash option to hide that warning message(!) but you can currently use the -v / --verbose option to learn more about what it is bringing in @dorab#2020-03-2704:01seancorfieldOK, I've repro'd based on your deps.edn and it looks like each of the three Jackson libraries being pulled in by Cheshire all contain that same file.#2020-03-2704:08seancorfieldYup, seems like each of the Jackson JAR files (core, databind-cbor, and databind-smile) all contain that file and it has a class name in it that matches one in the library it comes from. I'm fairly certain this is harmless and "expected behavior". Perhaps @ghadi can confirm when he is next around, since he's the original author of the first version of depstar...?#2020-03-2704:09seancorfield(this seems like another reason to try using clojure.data.json instead of cheshire 🙂 )#2020-03-2704:17dorabThanks @seancorfield. I will try to use clojure.data.json instead.#2020-03-2704:21dorabIf the same file is being requested multiple times, could depstar just include it once? Sorry if I'm being naive.#2020-03-2704:31seancorfieldNo, the same named file is included in three different libraries.#2020-03-2704:32seancorfieldAs I said above, Cheshire pulls in Jackson, and that means you get its core, cbor, and smile libs (dataformat, not databind, my bad -- rest of it still applies). Each of those three libs includes the same file. depstar concatenates the lines in each file to create a single file in the resulting JAR.#2020-03-2704:36dorabGotit. Thanks.#2020-03-2704:37seancorfieldInterestingly, dataformat-smile and dataformat-cbor are both deprecated and archived. Ah, looks like dataformat-binary is the replacement for both of those now. Sheesh, that is one weird, troublesome library...#2020-03-2715:47fabraoHello all, how to convert this lein uberjar
:main ^:skip-aot app.core
:uberjar-name "project.jar"
:profiles {:uberjar {:aot :all}}#2020-03-2715:47fabraoto deps.end alias?#2020-03-2716:31seancorfield@fabrao https://github.com/seancorfield/depstar#2020-03-2716:31seancorfieldIt has an example in the readme
# generate pom.xml (or create it manually)
clojure -Spom
# build the uberjar with AOT compilation
clojure -A:depstar -m hf.depstar.uberjar MyProject.jar -C -m project.core
# Main-Class: project.core
java -jar MyProject.jar
#2020-03-2716:31seancorfield(use project.jar instead of MyProject.jar and app.core instead of project.core)#2020-03-2716:33seancorfieldYou'll need to edit the pom.xml generated by clojure -Spom to add more stuff. Take a look at a pom.xml file generated by clj-new (if you created your project with a recent version of clj-new and the app template, then the pom.xml file will probably be almost 100% of what you would need for depstar and deployment to Clojars anyway).#2020-03-2717:22fabrao@seancorfield I tried -A:uberjar from your dot-clojure but I didn´t know about pom.xml#2020-03-2717:23fabraoso I creat project.clj to generate jar#2020-03-2818:21rgmapologies if this is in the docs, but is there a way to give a path to a deps.edn, instead of assuming it's in the current working directory?#2020-03-2818:22rgm(I'm coexisting with a bunch of Python and I'd prefer to have one deps.edn someplace else and just use long Makefile commands to get at the clojure libs)#2020-03-2818:25alexmillerno#2020-03-2818:34rgmdrat.#2020-03-2818:34rgmsymlinks it is ¯\(ツ)/¯#2020-03-2818:36rgm(or rather, I'll just cd other_place && clojure ... to keep my Python colleagues from tripping over the clojure stuff).#2020-03-2818:40seancorfield@rgm One thing you can try is specifying CLJ_CONFIG=/path/to/folder clojure ... where your deps.edn file is /path/to/folder/deps.edn -- this treats that deps.edn file as the user-level one, ignoring your ~/.clojure/deps.edn (and it would still pick up a deps.edn file in the current folder).#2020-03-2818:40seancorfield(so you'd get system deps, CLJ_CONFIG deps, current/project deps)#2020-03-2818:41rgmoh, that's a great trick ... I don't have a ton of deps#2020-03-2818:41rgmdon't really even need a current folder one.#2020-03-2818:41rgm(which is what got me going down this path, to keep things tidy).#2020-03-2818:41seancorfieldWe use this approach in our monorepo at work, so we can have a "standard" user-level deps plus our current subproject deps (and ignore any developer-level user deps on different machines).#2020-03-2818:46rgmright, and because it's env, I can just manage it through settings in Makefile, or .env, or direnv, or ... this is great.#2020-03-2818:54rgm(Can I just say you and Alex are killing it, and it’s much appreciated?)#2020-03-2819:13borkdude@rgm fwiw deps.clj (an unofficial clj script re-implementation, or the binary version deps.exe) does have an option for this#2020-03-2819:13borkdude@rgm fwiw deps.clj (an unofficial clj script re-implementation, or the binary version deps.exe) does have an option for this#2020-03-2819:14borkdudehttps://github.com/borkdude/deps.clj#extra-features#2020-03-2819:41rgmI hereby include you in the "killing it" set.#2020-03-2919:04joseis there an example about how to add a java file to my clojure project? In leiningen there is a :java-source-path , but I think there is nothing similar for tools-deps#2020-03-3005:20mike_ananev@U4NGX0FHN also you may try this https://github.com/redstarssystems/pbuilder#2020-03-3013:38josethanks, I didn't know about pbuild, maybe add it to https://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging ? I think I found an error on pbuilder docs, I'm opening a github issue#2020-03-3000:16sogaiu@jlle iiuc, compiling java is not in the scope of tools.deps -- may be something under here might be relevant? https://github.com/clojure/tools.deps.alpha/wiki/Tools#packaging#2020-03-3000:32potetm@jlle You can also call javac from the cli.#2020-03-3013:34joseI found that badigeon / pbuilder are good for my use case#2020-03-3014:12borkdude$ clj -Sdeps '{:aliases {:foo {:main-opts ["-e" "(ns,foo,(:require,[clojure.java.shell,:refer,[sh]])),(sh,\"javac\",\"Foo.java\"),(shutdown-agents)"]}}}' -A:foo
😜#2020-03-3014:13borkdude(it took me only 10 minutes to figure out the correct nesting and seancorfield-commas on the command line)#2020-03-3109:54borkdudehttps://theronic.github.io/depsy/#2020-03-3116:00jjttjjIs there a way to set the filename clash strategy with depstar? I'd like my data_readers.cljc to be merged#2020-03-3116:00jjttjjActually, it looks like that is done for the .clj but not cljc https://github.com/seancorfield/depstar/blob/9860bd5b510418fcc24d2660c8526bec7d53e7c1/src/hf/depstar/uberjar.clj#L61#2020-03-3116:48seancorfieldFeel free to open an issue on GitHub to add support for data_readers with additional file extensions. Seems like a good idea to add that. @U064UGEUQ#2020-03-3117:12jjttjjWill do!#2020-04-0108:34pezWhat is the equivalent of Leiningen's :local-repo "local-m2" in deps.edn?#2020-04-0108:38delaguardo:mvn/local-repo key#2020-04-0108:58pezIt doesn’t seem to work as I expect it to. I don’t know what to put as the url. #2020-04-0112:18alexmillerIt takes a string (a path)#2020-04-0112:19alexmillerWhich can be either relative or absolute#2020-04-0112:21pezThanks. I shall experiment some more, maybe I am not looking at the right places for the right result... Also, right now I'm using lein-tools-deps so maybe that complicates things.#2020-04-0112:22alexmillerI suspect that probably won’t work#2020-04-0108:38kszabohttps://github.com/practicalli/clojure-practicalli-content/issues/24#2020-04-0112:21alexmillerThis seems wrong as it is relative to the current directory. So if you are sharing it, you need an absolute path#2020-04-0111:19cursorkI think I'm just having a stupid moment... But I'm not sure where I can use :jvm-opts in deps.edn . I'm currently using it in a dev alias and calling clj -A:dev which works fine. But the entire purpose of this code is to interoperate via JNI to an underlying dynamic lib, so ideally I'd like to have :jvm-opts ["-Djava.library.path=..."] always be set to a path I control.
Using System/setProperty doesn't seem to have an effect as I guess the initial java.library.path at startup is used.
Is the only option to have an alias for the path, and always invoke it? And then other aliases as usual?#2020-04-0111:34cursorkMaybe I'm answering my own question... If there's a .dylib or .so I'm expecting to use, the location should be specified for each env it's deployed on. As it could be different between developer's machines as well. So maybe we all have a ~/.clojure/deps.edn with some profile with the correct path. A common name of :shared-libs or something might make that easy to work with.#2020-04-0111:44kszaboyeah, jvm-opts only works with aliases#2020-04-0111:46kszabohttps://clojurians-log.clojureverse.org/tools-deps/2018-11-12#2020-04-0111:46kszabo> https://clojurians.slack.com/team/U064X3EF3https://clojurians-log.clojureverse.org/tools-deps/2018-11-12/1541986798.430300It’s missing, no philosophical objection#2020-04-0111:47cursorkThanks for the confirmation 🙂#2020-04-0212:59rfhayashiHi! I'm trying to invoke tools.deps directly from clojure (more specifically clojure.tools.deps.alpha/resolve-deps) but I get a
Cannot access s3p://... using the registered transporter factories: HttpTransporterFactory, S3TransporterFactory, FileTransporterFactory
error. If I run a clj -Sdeps with the same content it works fine. Is there any library that I should include in the classpath to make s3 repos work? Or any environment variable or system property that I should set? I'm using org.clojure/tools.deps.alpha {:mvn/version "0.8.677"}.#2020-04-0213:06alexmiller@rfhayashi did you try s3:// ? that's the only documented protocol for s3 access (s3p did work in some older versions but has never been supported)#2020-04-0213:07alexmilleryou should not have to do anything special to get s3 to work#2020-04-0213:16rfhayashi@alexmiller it does work. thanks!#2020-04-0213:54kszabois there any support for modifying the bootclasspath, like:
https://github.com/sunng87/lein-bootclasspath-deps
does for leiningen? It would be nice to use some special case of :boot-deps so that I don’t have hack it together with downloading jar files locally from maven and using :jvm-opts directly#2020-04-0213:54kszabothis seems to be in the focus area of tools-deps of invoking java with the correct dependencies ready#2020-04-0214:02alexmilleryou should not modify the bootclasspath in my opinion, so I'm not going to make that any easier#2020-04-0214:06kszabothere are legitimate use-cases, like enabling alpn for jdk8 for http2:
https://www.eclipse.org/jetty/documentation/current/alpn-chapter.html
I agree that it should be avoided when possible, but now I need to hack around deps.edn because of the lack of support#2020-04-0214:14alexmilleryou can make an alias that produces the boot classpath you need, then use clj -Spath to generate the classpath from it, so there is no need for manual jar downloading#2020-04-0214:14alexmillerand then put that path in your :jvm-opts#2020-04-0214:15kszaboclj -Spath would produce an absolute path to my main maven repo, which is hosted in my home directory#2020-04-0214:16kszabomeaning the resulting :jvm-opts needs to be scripted on each machine#2020-04-0214:16alexmilleran alternative would be to set :mvn/local-repo to fetch it into a subdirectory of the project, then use relative paths in the jvm-opts#2020-04-0214:17kszabowouldn’t that apply to all maven deps? I haven’t used that one yet#2020-04-0214:17kszaboI don’t want to download maven again 😂 #2020-04-0214:18alexmilleryou would just use this for what you need in the bootclasspath#2020-04-0214:19kszaboI see#2020-04-0214:19kszabothanks for the help#2020-04-0221:29Jaime SangcapI'm using clojure cli / deps.edn. Is there a way to define env vars following the 12 factor app?
Looking into https://github.com/weavejester/environ but it seems like it does not support deps.edn (for instance, if your using lein. you can have .lein-env)#2020-04-0221:35seancorfield@jaime.sangcap (System/getenv "varname") is all you need to get an environment variable's value so I'm not sure what you're asking in the context of CLI/`deps.edn`?#2020-04-0221:36seancorfieldPerhaps "Can you have environment variables in your deps.edn?" No. It's a pure EDN file, not executable code, and has no substitutions made.#2020-04-0221:43Jaime SangcapI'm wondering if there is library that can load env var from file. And you can configure where the file is in the deps.edn.
Right now I'm only using intellij to set env vars.
What I'm thinking is there should be a example file (checkin to version control) that has all the required env var
Then locally, the library will check the example file against your env file for missing env var#2020-04-0315:35jeff.terrellThis might be at least partially what you're looking for: https://github.com/yogthos/config#2020-04-0316:21Jaime Sangcapcool! Yes I think can use this. Thanks 🙂#2020-04-0221:44Jaime SangcapBefore loading them and throw error is you missed to set something#2020-04-0221:44seancorfieldSorry, I have no idea what you mean by any of that.#2020-04-0221:45Jaime SangcapThanks! I think I'm mixing different things#2020-04-0221:46seancorfieldEnvironment variables are set at the O/S level before a process is started -- I would imagine IntelliJ is setting the environment variables before starting the REPL or run process.#2020-04-0221:47seancorfieldYou can't set environment variables from inside a process. You can set JVM properties tho'...#2020-04-0221:49seancorfieldBTW environ is just a library. It can be used from any process run by CLI/`deps.edn`.#2020-04-0221:49seancorfieldIt just happens to have existing plugins for lein and boot that call it in particular ways.#2020-04-0221:54seancorfield(but I would caution you against storing anything in deps.edn that isn't part of the official syntax -- deps.edn is not intended to be a "general configuration file -- which project.clj and build.boot tend to be)#2020-04-0307:38practicalli@jaime.sangcap Storing environment variables in a file that you then version control is the best way to add a huge security hole in your application and isn't following 12 factor app approach.
If you are using a service like Heroku, then that service manages the Environment variables securely for each application and you can the just read them in from Clojure as Sean mentioned or using environ.#2020-04-0308:56Jaime SangcapI agree. Thank you for pointing it.
However, what I meant was commit in version control an example file. Let's say .env.example , this will contain all the env vars that are needed by your application.
APP_PORT=8000
DATABASE_URL=
AWS_ACCESS_KEY_ID
SOME_PATH=/temp
Notice that env vars don't have values, but are there so that when other devs can copy the .env.example and create .env and provide the values.
I think an advantage of this one is you can check the .env file if there are missing keys from .env.example and throw an error before the app starts to prevent it from running in unpredictable state.#2020-04-0307:41teodorluOption: store config defaults in an .edn file, and create a custom read-config function that reads an env var if it is present, otherwise defaults to the file.
Then you can set defaults, and override them with environment variables.
Note: env vars are named by strings. We often like to use keywords as identifiers in Clojure. You might want to set a naming scheme that lets you query with keywords rather than strings.#2020-04-0308:28dominicmSounds like aero :)#2020-04-0309:37teodorluAero seems to make all of this possible to do from within the config file. README[1] excerpt:
{:port #or [#env PORT 8080]}
[1]: https://github.com/juxt/aero#2020-04-0309:52Jaime Sangcap@seancorfield Yes I think I got that one. Given I have this deps.edn
{:deps
{org.clojure/clojure {:mvn/version "1.9.0"}}
:paths ["src"]
:aliases {:dev {:extra-paths ["dev"]
:extra-deps {nrepl/nrepl {:mvn/version "0.6.0"}}
:main-opts ["-m" "nrepl.cmdline"]}}}
I could imagine intellij when you add env var DATABASE_URL to it, it will just run using DATABASE_URL=jdbc-url clj -A:dev
Looking at environ code, all functions are private 🙂
Maybe I could just do this. (copied from this convo https://github.com/weavejester/environ/issues/14)
(require '[environ.core :as environ])
(defonce env (merge (read-my-custom-config-file) environ/env)
Where config file is in .gitignore and will be used in local dev.
Thanks to @teodorlu for the idea#2020-04-0312:35alexmillerI’ve done something very similar to that in a real app#2020-04-0313:28jjttjjIs there a "recommended" way to have a repl start in a particular namespace and load that file?#2020-04-0313:29jjttjjI've been trying various arrangments of this but none seem to really be what I'm going for
:main-opts ["-i" "src/jt/jt15.clj"
"-e" "(in-ns,'jt.jt15)"
"-r"]#2020-04-0313:31jjttjjnevermind, had a typo in the namespace, this is working#2020-04-0400:14cflemingHow does deps handle snapshot versions? There doesn’t seem to be any setting for how often repos should be checked for new snapshots. Is there even any concept of a snapshot version in deps?#2020-04-0400:44seancorfieldI thought the underlying Maven library checked snapshots once a day?#2020-04-0400:45cflemingI’m not sure. It can be configured, using lein it can be set to :daily, :always or :never: https://github.com/technomancy/leiningen/blob/stable/sample.project.clj#L117#2020-04-0400:45cflemingDeps doesn’t seem to have a knob for that and I’m not sure what the default is.#2020-04-0401:09hiredmanI think snapshot builds may be fundamentally incompatible with how tools deps caches the built classpath#2020-04-0401:11hiredmanOr, I guess what is more likely is tools deps checks every time it needs to rebuild the cache, but if the classpath is cached and doesnt need rebuilding you'll never get a new snapshot#2020-04-0414:20alexmillerThis#2020-04-0401:21seancorfieldHmm, yeah, that makes sense. I guess the "knob" there is that you can -Sforce a rebuild of the classpath. I know Alex has said in the past that "RELEASE" and "LATEST" fly in the face of the stability that deps.edn is aiming for...#2020-04-0405:20cflemingYeah, I did wonder if deps might be philosophically opposed to them. If hiredman is correct that explains what some people are seeing in Cursive (very slow launches of REPLs) since Cursive doesn’t use the classpath caching.#2020-04-0413:46ghadiRELEASE & LATEST and origin/master are mutable references
SHAs are values#2020-04-0414:16alexmillerSnapshots are checked daily by default and there is no knob to change that atm#2020-04-0414:17alexmiller-Sforce will NOT force a recheck (although I’ve thought about making that flag also trigger update always so it would)#2020-04-0414:17alexmillerThe only real way to force a snapshot to update right now is to rm from your m2 repository AND -Sforce#2020-04-0414:19alexmillerI don’t see how snapshots would be any more likely to cause slowness than any other version#2020-04-0420:13cflemingThis can happen with lein if it’s configured to check for new versions always. That means every project resolve will hit the remote repo for every snapshot dep.#2020-04-0423:46alexmillerAh, yeah. Should just be daily w deps#2020-04-0419:39athomasoriginalCompiling a CLJS program and running a CLJS repl is straightforward enough with
clj --main cljs.main --compile ns.name.here --repl
However, if I want to run a cljs socket repl so I can connect my editor to it (Atom) I cam up with these two options
Option 1
; compile cljs
clj --main cljs.main --watch "src" --compile ns.name.here
; run a socket repl
clj -J-Dclojure.server.browser="{:port 4444 :accept cljs.server.browser/repl}"
Option 2
; do both in one command
clj -J-Dclojure.server.browser="{:port 4444 :accept cljs.server.browser/repl}" -- --main cljs.main --watch "src" --compile ns.name.here
I’m now wondering if there is a better option which I missed? 🙈#2020-04-0507:55didibusOne thing I once in a while wish for is a way to have tools.deps list out all of the available aliases.#2020-04-0507:57didibusLike it be nice if clj -A would list out the available alises instead of throwing an error. And same thing could happen for all other alias type, clj -R to list available resolve-deps alias, etc.#2020-04-0510:38borkdude@didibus since it's EDN it's very easy to do this yourself?#2020-04-0513:52alexmillerI like that idea#2020-04-0513:53alexmillerLet me think about the impl a little#2020-04-0517:11vlaaadI'm pretty sure Cursive does that#2020-04-0522:05cflemingYeah, Cursive shows all aliases and also whether they’re system or project ones. It doesn’t distinguish between system and user at the moment since the only way to determine that currently with deps is counting the config files, which is a little gross.#2020-04-0611:21pezI am trying to move a project over from Leiningen to deps. In the project.clj there is a top level :exclusionsvector. Anyone familiar enough with both lein and deps to hint me where I should look up how to translate it to deps?#2020-04-0612:27alexmillerThere is no support for global exclusions right now#2020-04-0612:27alexmillerBut you can repeat it at each top level dep if needed #2020-04-0612:32pezThanks! We'll see if needed. 😃#2020-04-0611:26vlaaadhmm, I think I saw something about using :override-deps with nil values…#2020-04-0611:30mgrbyteI am trying to figure out how to create an uberjar with exlusions (This achieved with the :provided profile in lein). I'm using depstar atm. Has anyone found a way to do this?#2020-04-0611:45rickmoynihan@pez: it’s translates directly:
some/dep {:mvn/version "3.0.0"
:exclusions [ch.qos.logback/logback-classic]}#2020-04-0612:22teodorluNote: deps uses different dependency resolution from Lein (which relies on Maven). It might make sense to check if you need the exclusions after moving to deps.
The ClojureScript podcast S3E4[1] with @alexmiller provides some discussion on this.
[1]: https://clojurescriptpodcast.com/#2020-04-0612:27imreWould it go against deps philosophy to add support for something like provided? To me that (although it could perhaps be better named) communicates for (some of) its functionality, this project needs xx/yy, however it is left to the consumer to provide those. Or is a dev alias with those dependencies the preferred way?#2020-04-0612:28alexmillerThe latter#2020-04-0612:30imreUnderstood, thanks#2020-04-0612:30pez@rickmoynihan yes, those exclusions work. This is a top-level key that Leiningen supports. I am not even sure what it does, only that I fail to get my project running and thinking it might be this. But after trying with @vlaaad’s suggestion, indicates I also have some other error in my config.#2020-04-0616:52seancorfield@pez FWIW, we used exclusions heavily with both lein and boot but once we switched to deps.edn we were able to get rid of nearly all our exclusions (I think we still have a couple of exclusions of ClojureScript in our backend deps because we just don't need/want that brought in).#2020-04-0617:10pez@seancorfield, Thanks! I'll listen to that podcast that @teodorlu mentioned and see if I can figure out why. 😃#2020-04-0621:08mikerodhttps://insideclojure.org/2018/05/04/add-lib/ was this feature never added?#2020-04-0621:11hiredmanhttps://github.com/clojure/tools.deps.alpha/tree/add-lib#2020-04-0621:24seancorfield@mikerod Recent commits have made add-lib depend on Java code in that repo so you can no longer depend on it via git. I built a local version to use, so I could work with the most recent features & fixes. Alex has talked about some form of add-lib ending up in core or part of the REPL or something... all TBD at this point.#2020-04-0621:25mikerodAh ok. Thanks for the details. #2020-04-0621:25seancorfieldMy "dot-clojure" file (`deps.edn`) has a :deps alias that depends on a git SHA prior to the Java code dependency.#2020-04-0621:26mikerodI did see that one. #2020-04-0621:26mikerodBut guess it requires some more. #2020-04-0621:27seancorfieldWell, it can be used as-is -- and it generally works just fine. I used it quite a bit in that form for loading new libraries into my running REPL (since I tend to start a REPL and run it for days).#2020-04-0714:49mikerodI believe I read there were issues in newer JVMs around trying to add to URL classloaders or something of that sort. This has been worked around in this branch? I’m going to look a bit at the impl because I’m curious what is being done.#2020-04-0714:50mikerodOf relevance is this https://github.com/tobias/dynapath#note-on-urlclassloader#2020-04-0714:50mikerod(which is used by pomegranate)#2020-04-0715:58seancorfield@mikerod If you create a dynamic class loader, you can load new libraries/classes into it: that's how we embed Clojure at work into a legacy app and how add-lib works (you can only use it in a context that has a DCL -- for example, Cognitect's REBL does not, so at work we have a small Clojure script to start REBL's UI after setting up a DCL).#2020-04-0716:04mikerodah ok. that makes sense - thanks for the extra details#2020-04-0621:28seancorfield(I'm using a variant of 0.8.667 built locally now)#2020-04-0621:36alexmillerI'm actually working on it RIGHT NOW :)#2020-04-0621:36alexmillerRich and I spent an hour this morning talking about it#2020-04-0621:38andy.fingerhutQuick. Everybody don't bother Alex! 🙂#2020-04-0621:44seancorfieldI deliberately didn't @ you so I wouldn't disturb you 🙂#2020-04-0622:02mikerodOh nice! #2020-04-0622:03mikerodOk. I appreciate all the input.#2020-04-0708:33penryuIs there anything in tools.deps that approximates lein's :java-source-paths?#2020-04-0708:36vlaaad@penryu no, tools-deps does not compile java, it only builds class paths and runs java programs#2020-04-0708:37vlaaadyou can compile java separately, release it to clojars and use as a maven dependency 🙂#2020-04-0708:38vlaaadthere are tools on top of tools-deps, like https://github.com/EwenG/badigeon — they can compile java sources#2020-04-0708:39penryuThanks, @vlaaad. In this case, I'm going to implement some of the core functions in java, and provide a clojure wrapper.#2020-04-0708:39penryuAh, I'll take a look at badigeon#2020-04-0709:28mike_ananev@penryu or you can take high level wrapper for Badigeon
https://github.com/redstarssystems/pbuilder#2020-04-0719:35teodorluIs it possible to override :main-opts from the command line?#2020-04-0719:46teodorluContext: standard :test:runner from Sean's clj -A:new setup. Trying to make Docker download test dependencies into the image without running stuff early on to improve caching.#2020-04-0720:01alexmillermain-opts concat rather than override#2020-04-0720:01alexmillerso no if I understand your question#2020-04-0720:02alexmillerbut you can always make a new alias that does what you want#2020-04-0720:08teodorluThat works. Thanks!#2020-04-1014:59borkdudedeps.clj, a community re-implementation of the clojure bash script in clojure, supports http proxy settings picked up from the environment:
https://github.com/borkdude/deps.clj#proxy-environment-variables
This was already supported for a while, but now it's documented.#2020-04-1021:47seancorfieldseancorfield/depstar {:mvn/version "1.0.94"} -- now merges data_readers for cljs and cljc extensions; adds -S/`--suppress-clash` option to suppress file clash warnings (some people find these information messages too annoying 🙂 ) https://github.com/seancorfield/depstar#2020-04-1021:48seancorfieldseancorfield/clj-new {:mvn/version "1.0.199"} -- mostly documentation updates; generated projects now use latest dependencies. https://github.com/seancorfield/clj-new#2020-04-1211:50afhammadHi, having issues importing java classes (generated by protobuf) from another folder in the monorepo using deps.edn . I've tried both adding the paths to :paths of my project and creating a deps.edn in the protos folder and adding that as a local dep. In both cases the classpath shows up when doing clj -Spath but starting a repl fails with ClassNotFoundException#2020-04-1212:10practicalli@afhammad have you tried {:local/root "/path/to/java/classes"} ? I used this to import java sources into a deps.edn project, using an alias, e.g. https://github.com/practicalli/clojure-deps-edn/blob/master/deps.edn#L63 Perhaps that will work for classes too, as it add them onto the classpath.#2020-04-1213:30afhammad@jr0cket does it have to be a zip? It's currently a folder with .java files. This worked using boot's :source-paths https://github.com/boot-clj/boot/wiki/Boot-Environment#env-keys#2020-04-1213:31mpenethttps://clojure.org/guides/dev_startup_time there are some hints here#2020-04-1213:51alexmillerIf you have Java files, you need to javac compile them into classes#2020-04-1213:52alexmillerIf you have classes, what you say above should work, but make sure the classes are in a directory structure that matches their package name (just like Clojure source)#2020-04-1219:11afhammadThanks @alexmiller. Precompiling worked. Is there a nice way of doing it as part of the deps.edn setup? One reason is it requires passing in a .jar dependency in order to compile which is already a dep of the project, but having the javac compilation as a separate step means I have to worry about fetching that dependency myself.#2020-04-1219:26alexmillerit will always be an extra step, but there are some helper dep tools out there to do that#2020-04-1219:27alexmillerone option is to portion the java stuff into its own project that you can build (with mvn or something else), deploy it, and depend on it#2020-04-1219:29afhammadyeh, thats one way to go, thanks#2020-04-1607:05flipmokidHi. What's the best way of adding java source to a deps.edn project? I have one java file that will be used as the entrypoint to the code that will call into the clojure code. It's there because Azure Function lib uses reflection on the method's arguements to determine how to convert data before passing it in and looks at the parameterised types used.#2020-04-1607:09andy.fingerhutI do not know the best way, as I haven't personally used this in a project before. I can tell you that this project claims to support the compilation of Java source files in a project that uses deps.edn. Exactly how, and whether it suits your purposes, is best determined by you: https://github.com/EwenG/badigeon#2020-04-1609:42flipmokidThanks @U0CMVHBL2 I'll give it a try.#2020-04-1611:31mike_ananev@U0C6HAPFU or you can try wrapper for it
https://github.com/redstarssystems/pbuilder#2020-04-1616:38flipmokidThanks @U097654L8, that looks really good.#2020-04-1810:39Eamonn SullivanHi all, I have a small experimental Clojure library (https://github.com/eamonnsullivan/github-search) that I'd like to turn into a mono-repo that produces both a jar and an npm package (shadow-cljs), with some common code (.cljc). Is this possible in tools-deps? Has anyone tried this and can point me to an example I can study?#2020-04-2107:08rbergerWhat would the equivaliant to the function leiningen.core.main/abort be in a program started by clj tool? My googling for a solution has been unsuccessful.#2020-04-2107:17dominicmI don't know the function, but I'd guess System/exit or throw based on the name.#2020-04-2107:18dominicmWhat behavior do you want?#2020-04-2107:22rbergerI guess Print msg to standard err and exit with a value of 1. And propler terminate anything and supply proper messages... I did use System/exit for now. I’ve been translating software that use to use lein to use tools.deps toolchain and looking for things like this.#2020-04-2111:15practicalliI have shared my user level ~/.clojure/deps.edn file, which includes aliases for all the common tools I use with Clojure projects. Instructions for using the aliases are in the README and in the relevant :alias sections. For some aliases that have optional arguments I have also created additional alias names to capture the most common options, to save the amount of details that need to be remembered. Any other useful aliases are most welcome and thanks to Sean Corfield for his collection of aliases.
https://github.com/practicalli/clojure-deps-edn
Thank you.#2020-04-2116:00seancorfield@jr0cket Is yours a superset of mine at this point, or is a selection of mine (plus your own additions)? Maybe I'll mine yours for extra stuff to add to mine 🙂#2020-04-2117:26practicalliThere is a substantial cross-over of libraries includes in the aliases. You have some I have not tested yet. I have some not included.
There is some divergence in naming of aliases.
Happy to work on convergence if there is interest / value.#2020-04-2119:35jrwdunhamHi all. Looking for some help here. I'm using tools-deps and my deps.edn file has a :repl alias with an :override-deps that we can summarize as {tegere {:local/root "/path/to/tegere" :deps/manifest :deps}}. The problem seems to be that the transitive dependencies of my local deps are not present in my classpath. I can see by running clj -R:repl -Stree that I have a dependency path like tegere/tegere > com.cemerick/pomegranate 1.1.0 > org.apache.maven/maven-resolver-provider 3.5.3 > javax.inject/javax.inject 1. However, if I ls -l /root/.m2/repository/ I can see that my top-level :deps dependencies are present but the dependencies of my local [:repl :override-deps] (e.g., javax.inject, pomegranate) are not. Any advice? Thanks in advance.#2020-04-2119:47alexmillercan you provide your clj version (`clj -Sdescribe`), a deps.edn to repro, and clj -Stree output?#2020-04-2119:57jrwdunhambash-5.0# clj -Sdescribe
{:version "1.10.1.502"
:config-files ["/usr/local/lib/clojure/deps.edn" "/root/.clojure/deps.edn" "deps.edn" ]
:config-user "/root/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/root/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
bash-5.0# cat deps.edn
{:deps
{cheshire {:mvn/version "5.8.1"}
clojure.java-time {:mvn/version "0.3.2"}
clj-http {:mvn/version "3.10.0"}
com.pubnub/pubnub-gson {:mvn/version "4.25.0"}
com.taoensso/timbre {:mvn/version "4.10.0"}
nrepl/nrepl {:mvn/version "0.6.0"}
cider/cider-nrepl {:mvn/version "0.22.0"}
hickory {:mvn/version "0.7.1"}
org.clojure/tools.namespace {:mvn/version "1.0.0"}
vbclient
{:git/url "#2020-04-2119:57jrwdunham#2020-04-2120:03alexmillerand clj -Spath doesn't include pomegranate and javax.inject?#2020-04-2120:06alexmilleroh, did you try -Sforce to force re-create your classpath?#2020-04-2120:06alexmillerchanges in local deps' deps.edn files do not trigger a classpath cache refresh so you might be seeing something old there#2020-04-2120:10jrwdunhamThose deps are present when I run clj -Spath#2020-04-2120:10jrwdunham$ clj -R:repl -Spath
src:/root/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:...
#2020-04-2120:11jrwdunhambut there is nothing at those locations on disk#2020-04-2120:11jrwdunhambash-5.0# ls /root/.m2/repository/javax
ls: /root/.m2/repository/javax: No such file or directory
#2020-04-2120:19jrwdunhamOdd, if I run clj -R:repl -Sforce -Stree then it seems to work...#2020-04-2120:19jrwdunhambash-5.0# ls /root/.m2/repository/javax
inject
#2020-04-2120:20jrwdunhamthing is, I'm trying to build a Docker image, and having the following RUN statements in my Dockerfile is not having the same effect:
RUN clj -R:repl -Sforce -Stree
RUN clj -R:repl -Sforce -Stree
#2020-04-2121:23jrwdunhamWell, after removing the .cpcache/ dirs on my host machine and ensuring I have a Docker COPY that ensures my local :override-deps paths are available at build time (d'oh!), and running a Docker build with --no-cache, all seems to be working as expected.#2020-04-2121:24jrwdunhamThanks for your help @U064X3EF3 (even if the issue seems ultimately to have been my Docker misconfiguration)—I at least learned more about how to use tools-deps, and maybe this will help somebody else.#2020-04-2121:26alexmillerif you have a cached classpath, it does not currently verify that the dirs/files on the classpath actually exist, so manual mucking with ~/.m2 (either by the filesystem or your container config) can lead to some confusing results. -Sforce will usually help you there#2020-04-2121:26jrwdunhamcool. gtk#2020-04-2121:27alexmillerit's a little tricky to implement this check as it would need to be in the bash script#2020-04-2121:27alexmillerand it requires some interpretation b/c missing path dirs is fine and pretty common#2020-04-2216:32unbalancedOkay finally have my private maven all set up on artifactory, verified that the code is deployed there. Choking on how to include that in the deps.edn for the project, though.
:deps
{com.business.projname {:mvn/version "0.1.5"
:mvn/repos {"central" {:url ""}}}
I assume that is not correct#2020-04-2216:33unbalancedon
$ clj -m figwheel.main -bo dev
Error building classpath. Could not find artifact com.business.projname:com.business.projname:jar:0.1.5 in central ()
#2020-04-2216:36unbalancedah, think I got it. well at least it's complaining it can't file the file at the right location#2020-04-2216:36unbalancednew error message, I'll call that a success#2020-04-2216:37imreIsn't :mvn/repos supposed to be a top-level key?#2020-04-2216:38unbalancedyep. and I needed so use deps correctly#2020-04-2216:39unbalanced:mvn/repos {"business" {:url ""}}
:deps {com.business/projname {:mvn/version "0.1.5"}}
#2020-04-2216:39unbalancednow at least I get a new error with that#2020-04-2216:39unbalanced😄#2020-04-2216:41unbalancedah okay, no error message now, works exactly as desired#2020-04-2216:41unbalancedyay#2020-04-2216:41unbalancedsuspiciously painless#2020-04-2306:11ikitommiwhen adding dependency using :git/url, using clojure.tools.namespace.repl/refresh seems to refresh also namespaces from those dependencies. Is there a way not to do that?#2020-04-2306:12ikitommiadded dependency to malli and it pulled sci, refreshing a lot of non-project ns’es:
:reloading (sci.impl.namespaces sci.impl.utils sci.impl.destructure sci.impl.fns sci.impl.interop sci.impl.doseq-macro sci.impl.exceptions sci.impl.max-or-throw sci.impl.for-macro sci.impl.analyzer sci.impl.readers sci.impl.parser sci.impl.interpreter sci.core malli.core malli.provider malli.json-schema malli.transform malli.edn malli.error malli.mermaid malli.swagger user sci.impl.main malli.util malli.generator)
#2020-04-2306:25alexmillerI don't know enough about refresh to say#2020-04-2306:26alexmillerI think refresh looks at directories on the Java classpath, and git deps will cause dep paths to be there#2020-04-2306:27alexmillerrefresh doc string says "The directories to be scanned are controlled by 'set-refresh-dirs'; defaults to all directories on the Java classpath."#2020-04-2306:27alexmillerso you could use that to narrow the scope#2020-04-2306:28alexmiller(clojure.tools.namespace.repl/set-refresh-dirs "src")
#2020-04-2306:28alexmillermight be sufficient#2020-04-2306:28ikitommiwhen using :mvn dependency, the code is in jars and because of that, not in scope of refreshing?#2020-04-2306:29alexmillerright#2020-04-2306:30ikitommiset-refresh-dirs did the trick, thanks!#2020-04-2306:30alexmillerwhen in doubt, read the docs :)#2020-04-2306:50ikitommiwould be great if this worked out-of-the-box, e.g. convention from DEPS to which folder the pulled deps are in, that is automatically excluded in TNS.#2020-04-2307:06alexmillerby "box" I assume you mean tools.namespace?#2020-04-2307:07alexmillerI think I would invert it - only include src dirs that are relative to the current directory and exclude others#2020-04-2307:07alexmillertns doesn't know anything about deps or git deps or whatever#2020-04-2308:29holyjakHi! Is there support for adding an optional dependency, without specifying its version? clj-http has an optional dependency on cheshire and I would like to include it, using whatever version this version of clj-http wants. So instead of cheshire/cheshire {:mvn/version "5.8.1"} I want just cheshire/cheshire . Is this possible? Thank you!!!#2020-04-2311:27borkdudeI don't think it's possible since clj-http doesn't prescribe the cheshire version as a dependency, it's only available at development time for clj-http I think?#2020-04-2312:28alexmillerNo, this is not possible, you’ll need to pick a version#2020-04-2314:55Laverne SchrockI feel like I've heard mention of others running into this:
Cloning: <actual repo name snipped>
Error building classpath. Destination path "ops-cli" already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path "ops-cli" already exists and is not an empty directory
at org.eclipse.jgit.api.CloneCommand.verifyDirectories(CloneCommand.java:257)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:189)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:49)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
at clojure.tools.gitlibs.impl$ensure_git_dir$fn__955.invoke(impl.clj:109)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
at clojure.tools.deps.alpha.extensions.git$eval999$fn__1001.invoke(git.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:169)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:237)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
Unfortunately, when I was trying to figure out what the reproduction steps were, the problem suddenly went away. I can however say, that before it started working I was able to confirm that ~/.gitlibs/libs/.../ops-cli existed with a few SHA directories, but not the one corresponding to the one I was using in my deps.edn. This was running version 1.10.1.492 of clj/clojure on Linux.
Would it be possible to add some logging here: https://github.com/clojure/tools.gitlibs/blob/master/src/main/clojure/clojure/tools/gitlibs/impl.clj#L109 so that in the future we can see what the original Throwable was?#2020-04-2315:01alexmillerI have some good guesses at this point what the problem is#2020-04-2315:02alexmillerI think it's related to having two threads/processes cloning the same lib in parallel#2020-04-2315:03alexmillerbecause deps expansion is now parallel this is much easier to encounter now#2020-04-2315:03alexmillerand/or it could even just be bugs in the much older version of jgit we're using#2020-04-2315:04alexmillerI have a long paused set of work in a branch to update to latest jgit etc and I wanted to do that first#2020-04-2315:05alexmillerif it is the parallel stuff, you can set -Sthreads 1 to turn it down#2020-04-2315:53Laverne SchrockI'll give that a try if I run into it again#2020-04-2315:06alexmillerprobably longer term to actually "fix" this, needs more thought#2020-04-2405:44lispycloudsHello, what is a recommended/nice way of organizing a mono-repo like project of multiple tools.deps backed projects in terms of being able to run all tests and produce JARs/docker images/run tests for each from the root of the project? I have something like this, using Makefiles:
.
├── Makefile
├── docker-compose.yaml
├── consumer
│ ├── Dockerfile
│ ├── Makefile
│ ├── deps.edn
│ ├── src
│ │ └── queue
│ │ └── consumer.clj
│ └── test
│ └── queue
│ └── consumer_test.clj
└── producer
├── Dockerfile
├── Makefile
├── deps.edn
├── src
│ └── queue
│ └── producer.clj
└── test
└── queue
└── producer_test.clj#2020-04-2406:33dominicm@rahul080327 I'd recommend looking at juxt edge which is my implementation of a monorepo. If you are looking to share makefiles, you probably want the include instruction in make (not shown in edge)#2020-04-2406:53lispycloudsThis looks really neat @U09LZR36F!
I was considering going the shell approach but was thinking the need for manual orchestration and error handling might be complex hence opted for Make. But will try out your organized approach! Thanks! 😄#2020-04-2406:34dominicmIn edge I solve the reuse problem with bin scripts at ../bin/eftest, etc#2020-04-2413:58zaneIs it possible to use clj and friends private repositories via a password-protected SSH key?#2020-04-2414:00ghadithere's no interactive ssh with the tools @zane#2020-04-2414:00ghadissh-add the key to your running ssh-agent#2020-04-2414:00ghadithen you can run clj#2020-04-2414:02zane@ghadi Yes, I did that but I’m getting an error (`USERAUTH fail`) when I try it with a password-protected SSH key.#2020-04-2414:03zaneWorks when I use a SSH key that does not have a password.#2020-04-2414:03ghadiif you run ssh-add -l you see the key you intended to add?#2020-04-2414:04zaneI believe I added it, yes. Let me run through it one more time to be sure.#2020-04-2414:04zaneBut it sounds like what you’re telling me is that this ought to work. 🙂#2020-04-2414:04alexmillerI don't think it should work#2020-04-2414:05alexmillerI don't think password protected ssh keys are supported in the set of stuff we're running#2020-04-2414:05zaneAh, okay.#2020-04-2414:05ghaditotally should work#2020-04-2414:05ghadiit prompts for pw when adding to the ssh-agent#2020-04-2414:05ghadiit's non-interactive after that#2020-04-2414:06ghadiif you see the key in your agent, make sure it doesn't appear in ~/.ssh/config#2020-04-2414:06ghadiJSch, the underlying ssh library, is stupid w.r.t. understanding .ssh/config when there is also an ssh-agent present#2020-04-2414:07ghadi(It prefers to read the key directly from the file, if it knows about it)#2020-04-2414:07ghadi(Rather than consulting the agent)#2020-04-2414:08ghadiSo, what's in your ssh agent (`ssh-add -l`) and what's in your .ssh/config?#2020-04-2414:09zane% ssh-add -l
4096 SHA256:… /Users/zane/.ssh/id_rsa (RSA)
% cat ~/.ssh/config
Host *
AddKeysToAgent yes
UseKeychain yes
IdentityFile ~/.ssh/id_rsa
#2020-04-2414:09ghadiok comment out that block in your .ssh/config#2020-04-2414:09ghaditemporarily#2020-04-2414:09ghadithen run clj again#2020-04-2414:10zaneclj succeeds, but I think it’s pulling from a cache.#2020-04-2414:10zanei.e. It’s using the copy of the repository I downloaded when I was using a passwordless SSH key.#2020-04-2414:11ghadiremove that from ~/.gitlibs to verify#2020-04-2414:11zaneTrying to remember how to clear that cache.#2020-04-2414:11zaneRad. Thanks.#2020-04-2414:11ghadishould be in two places in there#2020-04-2414:11ghadi~/.gitlibs/{_repos and libs}#2020-04-2414:11zaneAny harm in just blowing away ~/.gitlibs entirely?#2020-04-2414:12ghadidunno how much you have in there#2020-04-2414:12ghadibut anything will be redownloaded if you have creds#2020-04-2414:12ghadino worse than blowing away ~/.m2/repository#2020-04-2414:13zaneAwesome.#2020-04-2414:13ghadiso did it work?#2020-04-2414:13zaneTrying.#2020-04-2414:13zaneStarted up instantly again.#2020-04-2414:13zaneThat’s odd.#2020-04-2414:14zanei.e. I didn’t see it download the private repository.#2020-04-2414:14ghadiclj -Sforce#2020-04-2414:15zaneWorked.#2020-04-2414:15ghadiNice.#2020-04-2414:15ghadi@alexmiller#2020-04-2414:15zaneThanks!#2020-04-2414:16zaneSo, what should I do with ~/.ssh/config now?#2020-04-2414:16zaneLeave that stuff commented?#2020-04-2414:16ghaditl;dr: https://clojurians.slack.com/archives/C6QH853H8/p1587737218326100#2020-04-2414:16zaneI confess I don’t understand SSH well enough fully understand the implications of that.#2020-04-2414:17ghadiI don't think in your case there is much harm in removing just the IdentityFile portion of that block#2020-04-2414:17zaneGot it. I had hoped that was the case.#2020-04-2414:17ghadibecause ~/.ssh/id_rsa is one of the default searched keys#2020-04-2414:18zaneBut for people with multiple SSH keys that might be an issue, sounds like?#2020-04-2414:18zane(I’m writing up documentation on how to do this for the rest of my team.)#2020-04-2414:20ghadiit depends#2020-04-2414:21ghadido you have 1 SSH key <> per repo?#2020-04-2414:21ghadior 1 SSH key that grants you access to a bunch of repos, but you also have unrelated SSH keys present too?#2020-04-2414:21ghadiThe former is not supported, the latter is, AFAICT#2020-04-2414:22zaneI personally only have this one key, but I imagine others on the team might have more.#2020-04-2414:22zaneI see.#2020-04-2414:22zaneAlso, I just want to confirm one more thing: https://ask.clojure.org/index.php/8725/deps-support-newer-private-file-formats-types-such-ed25519 that clj won’t work with OpenSSH-format keys and that you must use -m PEM when generating them. Is that correct?#2020-04-2414:23ghadiMy personal recommendation is to not use files at all, only put your ssh keys into the agent#2020-04-2414:23zaneI see. I’ll read up on how to do that.#2020-04-2414:24ghadiclj can use fancy newer keys like ed25519 keys as long as JSch is asking the agent to deal with them#2020-04-2414:24ghadiIf JSch is trying to read them from the key files directly -> no bueno#2020-04-2414:24zaneI see!#2020-04-2414:25ghadiall of this is a pain IMHO#2020-04-2414:25ghadigithub style https auth tokens are a simpler model, and offer finer grained authority#2020-04-2414:25zaneBut that’s not supported at the moment?#2020-04-2414:25ghadieasier rotation/revocation#2020-04-2414:26ghadiright, not supported#2020-04-2414:26zaneGot it.#2020-04-2414:26zaneThis is extremely helpful, @ghadi. Thanks again for your time.#2020-04-2414:26zaneI may try to write this up so others can benefit.#2020-04-2414:26ghadia buddy at Github told me <35% of the auth they see is ssh based#2020-04-2414:27zaneRight. I certainly wasn’t using SSH with GitHub until I had to get clj to work with a private repository.#2020-04-2414:38dominicmWeird, I always considered https the less-secure approach.#2020-04-2414:41ghaditokens can be ephemeral, ssh identities are long-lived#2020-04-2414:45ghadihttps://developer.github.com/v3/guides/managing-deploy-keys/#2020-04-2414:45ghadigood pros/cons ^#2020-04-2610:29thhellerclj -A:dev
Program 'java.exe' failed to run: The filename or extension is too longAt
C:\Users\thheller\Documents\WindowsPowerShell\Modules\ClojureTools\ClojureTools.psm1:353 char:5
+ & $JavaCmd @JvmCacheOpts @JvmOpts "-Dclojure.libfile=$LibsFile" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At C:\Users\thheller\Documents\WindowsPowerShell\Modules\ClojureTools\ClojureTools.psm1:353 char:5
+ & $JavaCmd @JvmCacheOpts @JvmOpts "-Dclojure.libfile=$LibsFile" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ResourceUnavailable: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : NativeCommandFailed#2020-04-2610:30thhellerwindows powershell. I'm assuming that means the classpath is too long?#2020-04-2610:52dominicmI know people have reported long classpath issues on windows.#2020-04-2610:52dominicmSomeone looked into copying whatever it is mvn does, I think it involved a file with the cp in.#2020-04-2611:09sogaiui think this may be related: https://clojure.atlassian.net/browse/TDEPS-120#2020-04-2611:11sogaiui've summarized a bit of what i found here: https://github.com/sogaiu/clj-pathing-jar#technical-background#2020-04-2812:36Bhouglandtotal newb question, but where does deps.edn install the libraries?#2020-04-2812:40kszabo~/.m2 and ~/.gitlibs#2020-04-2812:40kszaboby default#2020-04-2812:40teodorlu~/.m2 is the same as Maven and Leiningen.
$ find ~/.m2 -iname 'clojure*jar'
/home/teodorlu/.m2/repository/net/mikera/clojure-utils/0.8.0/clojure-utils-0.8.0.jar
/home/teodorlu/.m2/repository/clojure-msgpack/clojure-msgpack/1.2.1/clojure-msgpack-1.2.1.jar
/home/teodorlu/.m2/repository/org/clojure/clojurescript/1.10.520/clojurescript-1.10.520.jar
/home/teodorlu/.m2/repository/org/clojure/clojurescript/1.10.597/clojurescript-1.10.597.jar
/home/teodorlu/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/home/teodorlu/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar
/home/teodorlu/.m2/repository/clojure/java-time/clojure.java-time/0.3.2/clojure.java-time-0.3.2.jar
/home/teodorlu/.m2/repository/com/theoryinpractise/clojure-maven-plugin/1.7.1/clojure-maven-plugin-1.7.1.jar
/home/teodorlu/.m2/repository/com/cemerick/clojurescript.test/0.0.4/clojurescript.test-0.0.4.jar
#2020-04-2812:40Bhouglandthen the trace.edn tellls my project how to find them?#2020-04-2812:42Bhouglandnevermind. that doesn't show a path on my local machine#2020-04-2812:43teodorluBased on your deps.edn file, you get a classpath. Try running clj -Spath in a deps project, and you'll se it refer back into where your libraries are installed.#2020-04-2812:44Bhougland{:version "1.10.1.489"
:config-files ["/home/ben/.clojure/deps.edn" "deps.edn" ]
:config-user "/home/ben/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/nix/store/msn2kqpck9c5iin8hgjzm5chpwpm2167-clojure-1.10.1.489-prefix"
:config-dir "/home/ben/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}#2020-04-2812:44BhouglandDoes this look okay?#2020-04-2812:45BhouglandI am questioning the :install-dir because I use Nixos and this path would be immutable#2020-04-2812:48alexmillerThe install dir isn’t used anymore#2020-04-2812:45teodorluI must admin that I'm not sure about what you're asking.#2020-04-2812:46Bhouglandmy repl can never mind my classpath#2020-04-2812:47teodorluIf you want to use deps.edn the "Nix way", I'm guessing you'll want guaranteed immutable references to what's normally a mutable folder in ~/.m2. But then you'll have to do some major modifications to tools-deps, I think.
Other people might be able to answer better.#2020-04-2812:48BhouglandI don't really want to do it the nix way. I just want to transistion to deps.edn and my repl yells at me that it can never find my classpath#2020-04-2812:49teodorluOh, so you've installed the clojure tool via Nix, it doesn't work, and you just want it to work?#2020-04-2812:49BhouglandYes. But I am wondering if it is nix or my lack of understanding on some point#2020-04-2812:50teodorluHmm. I love Nix, but for Clojure I just use the official installer: https://clojure.org/guides/getting_started#_clojure_installer_and_cli_tools#2020-04-2812:52BhouglandI appologize, earlier I posted Sdescribe not Spath. Spath is showing the .m2 directiory like it should#2020-04-2812:53Bhouglandanother vairable that is probably tripping me up is that I wired together Kakoune and the klipse-repl. I probably did something wrong there.#2020-04-2812:54BhouglandThanks for the help#2020-04-2812:56teodorluDon't think I'll be able to help you with that setup. If you want a simple experience getting started, the official Clojure installer + #calva should be a good place to start.#2020-04-2812:57BhouglandThanks for the support, it helped me target the most likely issue.#2020-04-3018:34dominicm@alexmiller I know it's been discussed before, but one good reason to have access to the resulting deps.edn that built the current JVM is for file watching. I want to automatically configure figwheel to watch my :paths and my :local/root deps, but not my gitlibs (which are dirs on the classpath). Workaround is to filter anything containing ".gitlibs", but I'm not sure of the portability or future-proof of that (as soon as someone needs a config flag for that, the tower falls).#2020-04-3018:35alexmillerit's coming...#2020-04-3018:36alexmillerthat work is done and in a branch but ties into ongoing work on add-lib and tools.build and maybe even some clojure core stuff#2020-04-3018:39alexmillerthe new term we're using for this is the "runtime basis", which is a superset of the merged deps.edn that also includes your effective alias mods, the lib map and a (datafied) classpath. clj will compute this, cache the basis file, and inject a reference to it via a system property. we've got some additional apis to read and help you use the basis that may end up in clojure proper (the reading part does not depend on tools.deps - it's just data), but those are really just helpful, it's just reading edn and using clojure data#2020-04-3018:40alexmillerfyi, there is an env var that defines the gitlibs root, so you could base it on that (GITLIBS)#2020-04-3018:40dominicmAh, interesting. Didn't know that.#2020-04-3018:40alexmillerhttps://github.com/clojure/tools.gitlibs#cache-directory#2020-04-3018:40dominicmI've also just realized that the libfile is sufficient. I can figure out the intersection of classpath dirs & gitlibs from that.#2020-04-3018:41alexmillerso, that is currently being injected by clj as I describe above, but that's probably going to be replaced#2020-04-3018:41dominicmI've spent a lot of time this afternoon due to a collision of me trying to remove code that filtered libs out (but indirectly - so I didn't know that was the difference) & a bug figwheel where it can't load certain namespaces that are in watch dirs, and it's caused by a particular dependency we have... and now it's almost 20:00!#2020-04-3018:42alexmillerthere is a system property clojure.libfile that points to the libfile used to start the current process#2020-04-3018:42alexmillerassuming you start with clj of course#2020-04-3018:42dominicmHmm, okay. I'll probably make sure my code explicitly removes the bad dependency then, that way it'll continue working after the clj version bump.#2020-05-0122:43drewverleewhats the difference between
-Jopt Pass opt through in java_opts, ex: -J-Xmx512m
-Oalias... Concatenated jvm option aliases, ex: -O:mem
the first is jsut for one option and the later is for multiple?#2020-05-0122:48drewverleehttps://clojureverse.org/t/how-to-pass-jvm-opts-to-clj-from-terminal/1940/3?u=drewverlee#2020-05-0122:55hiredmanaliases are a feature of tools.deps, where you can kind of group alternative configurations together and give them a name, and -O lets you apply them#2020-05-0122:55hiredman-J is for passing arguments directly to the command that launches the jvm, typically 'java'#2020-05-0122:56drewverleeWhat would it look like to psas multiple arguments to the jvm? each one gets a J-<arg>#2020-05-0122:56drewverleeclojure J-<arg> J-<arg>#2020-05-0122:56hiredmanpassing arguments is bad wording on my part#2020-05-0122:57hiredmanit is passing options, flags like -Xmx512m#2020-05-0122:58hiredmanso for example, you could pass -J-server (which is the default these days)#2020-05-0122:58drewverleeso for multiple its like clojure ... -J-Xmx512m -j-Xms512#2020-05-0122:58seancorfield(! 613)-> clj -J-Dfoo=bar -J-Dquux=wibble
Clojure 1.10.1
user=> (System/getProperty "foo")
"bar"
user=> (System/getProperty "quux")
"wibble"
user=> ^D
#2020-05-0122:59hiredmanI would put them first#2020-05-0122:59seancorfieldOne -J per argument you want to pass. Beware of white space 🙂#2020-05-0122:59hiredmanwhen you run clojure or clj the argument handling is complicated#2020-05-0122:59drewverleenice. i didnt know about system/getProperty.#2020-05-0123:00hiredmansome arguments are for scripts, some for jvms, and some for clojure, and some for your code#2020-05-0123:02hiredmanclj/clojure also have another of different ways to combine aliases, but -A will usually do want you want#2020-05-0123:03hiredman(aliases can also specify jvm options, and -O combines those ignoring the other stuff specified in the aliases)#2020-05-0219:44Audriushello! where is source from the Jar file that is bownloaded by this installer https://clojure.org/guides/getting_started#_installation_on_linux ?#2020-05-0219:49seancorfieldIt's an uberjar built from this https://github.com/clojure/tools.deps.alpha#2020-05-0219:50seancorfieldThe installer's source is here https://github.com/clojure/brew-install (the shell scripts etc) and that readme talks about the components, including the uberjar above @stebokas#2020-05-0310:03victorbHello! I've got an idea for my own extension to tools.deps.alpha, I can see bunch of them in the src/main/clojure/clojure/tools/deps/alpha/extensions/ path. So let's say I now have my X extension written, how can I make tools.deps.alpha use it? I guess I want to end up with something similar to a lein plugin where installing it gives you the ability of using the new extension. I'm not super familiar with Java (mostly been dealing with ClojureScript) so don't expect anyone to give me the straight path to my solution, but some pointers/keywords for what to lookup would be most appreciated!#2020-05-0310:07dominicmUsing it with the library is easy, using it with the cli is not yet possible#2020-05-0310:10victorbI see. Thanks! So for now, if I'd like to publish it to enable others to use it, I would have to publish a forked version of tools.deps.alpha with my extension worked into the codebase?#2020-05-0310:15dominicmNo. They would just require your namespace which extends the multi method#2020-05-0310:18victorbHm, but if this is not exposed to do in the cli, it feels like a workaround at best, only for people who are using deps as a library. Not sure how the common use case looks for using deps (as I'm still mostly using leiningen), but feels like most people would still stick to using the cli. But again, I don't know enough about this to say for sure#2020-05-0310:32dominicmIn that case you'll need to fork the cli#2020-05-0310:34victorbalright. Thanks a lot for the explanations!#2020-05-0312:16alexmillerWhat’s your extension?#2020-05-0312:25victorb@U064X3EF3 fetching dependencies via a content-addressed database (IPFS or similar) and splitting versions into two different versions, one for the interface and one for the implementation#2020-05-0312:32alexmillerIncluding an extension would just require changing the one line in the clojure script that defined the tools classpath#2020-05-0312:33alexmillerWell, I guess the extension also needs to be loaded#2020-05-0312:35alexmillerclj has been out for 2 or 3 years and this is the first time someone has asked about this so it hasn’t really been a priority#2020-05-0312:50victorbyeah, if I'll fork tools.deps I can see the path forward easily, but for obvious reasons I wanted to avoid that.
Makes sense that no one asked, as it's a bit bleeding-edge stuff to load dependencies from something that is not HTTP/S
Thanks Alex!#2020-05-0313:05dominicmI think it gets more complex if you want to support dependencies which depend on an extension.#2020-05-0314:22victorbHm, yeah, might have to have some sort of bootstrapping available or ship whatever the extension comes with over clojars/maven. Unless I just end up with a external babashka cli util that does the fetching... Not sure yet, exploring the options#2020-05-0316:58alexmillerFiguring all that out is why we haven’t done it yet :)#2020-05-0315:44murtaza52what is the lein clean equivalent for cli tools ?#2020-05-0315:57dominicmclj -Sforce #2020-05-0317:03alexmillerThere isn’t anything to clean#2020-05-0317:04alexmillerForce will force a recompute of the cached classpath which is needed in a few special cases but rare#2020-05-0317:05alexmillerIn lein, there is a build directory with artifacts, but that’s not a thing in clj (YET!)#2020-05-0317:08dominicmDoes that mean it will be a thing in clj, or that it will be a thing in a tools.build main?#2020-05-0317:08dominicm(wondering how integrated these things are)#2020-05-0317:10vlaaadI hope tools.build will have some sha hashes or something for its inputs/outputs caching so it won't require cleaning...#2020-05-0317:14alexmillerNot going to get into any details now, sorry :)#2020-05-0317:16dominicmWe'll have to learn how to bribe you. I hear the ops team at facebook has a full wall of alcohol.#2020-05-0412:21tianshuHow can I make clojure cli download jars if Clojure 1.10.1 in ~/.m2 is not found.#2020-05-0412:27alexmillertry using -Sforce#2020-05-0412:28alexmillerthat will force the classpath to be recomputed, not used from cache and should trigger a download#2020-05-0412:31tianshu@alexmiller Thanks! it works#2020-05-0414:59drewverleeIs there a way to run the clj/clojure script so it doesn't try to resolve deps first? were getting a classpath error and were hoping to get some diagnostic information via J-XshowSettings:properties, but we hit the "cannot find artifact" error first.#2020-05-0415:07alexmillerif you don't resolve deps, you can't make a classpath, and thus can't start a JVM to use that setting, so no?#2020-05-0415:09drewverleeThat makes sense. I guess i need to oreint myself, clj assumes you can start a JVM, but for debugging their is some information you can get before that. So i can just run java -XshowSettings:properties.#2020-05-0415:10alexmilleris that a thing?#2020-05-0415:11drewverleeis what a thing? 🙂#2020-05-0415:11alexmillershowsettings#2020-05-0415:11alexmillerI see it is - was confused by the help message after nvm#2020-05-0416:25drewverleeIf a deps procurer (local/root) points to another deps with a Procurer (e.g mvn/repos) Can i assume that it will read that Procurer or does it just grab the deps?#2020-05-0416:35alexmillerif you're asking whether transitive dependency maven repos will be used to resolve dependencies, then no#2020-05-0416:35alexmillermaven allows this, but it seems awfully dicey from a security pov so I kind of consider it an anti-feature#2020-05-0416:46drewverleeits likely i'm not describing this correctly.
I have a deps.edn with {:deps {datomic {:local/root "../datomic"}} and in "../datomic" the deps reads {:mvn/repos {"http://my.datomic.com" {:url "https..."}} It seems i have to add the mvn repos to the deps with the local/root procurer, even though that deps doesn't directly have a dep on datomic-pro. It has that dependency through the other deps.
@alexmiller as always, thanks for the help.#2020-05-0418:00alexmillerYeah, that’s the scenario I’m describing#2020-05-0418:01alexmillerThe question is, do you want your transitive deps to make decisions about where you get things on the internet?#2020-05-0418:02alexmillerThe answer is definitely not an unqualified yes#2020-05-0418:02alexmillerSo it needs a lot of thought#2020-05-0418:10drewverleei understand. thanks alex.#2020-05-0420:05vlaaadokay, I know how to bait Alex Miller to tell us some more about tools-build, watch this:#2020-05-0420:06vlaaadtools-build is a fork of scala's SBT#2020-05-0420:27alexmillerdo you want me to write about it, or do you want me to finish it? :)#2020-05-0420:43dominicmI was going to make a joke reply, but now I'm trying to decide my actual stance on it... If I know the high level goal I can plan around it and start getting my chickens in a row 🐔#2020-05-0421:18seancorfield@dominicm I'm curious how your chickens would be organized differently based on various possible high level goals of tools.build?#2020-05-0507:35dominicmApparently my ducks would be because I was a bit daft last night#2020-05-0507:45dominicmAnyway. I usually find libraries from core require a shift in thinking or approach. Eg clj forces you to do your own builds, so I'd expect the ecosystem to start building -mains earlier in anticipation.
I have done my best to make sure my build functions can be used in different ways, but I'm betting I missed something.
Knowing whether it solves the java sources problem would allow people using lein currently who want to move off to start doing any prep they need, as I doubt there will be something as straightforward as :java-sources.#2020-05-0515:09seancorfieldI'm not sure what "prep" those lein users could realistically do, even if the goal was stated as "solves the java sources problem"?#2020-05-0515:13seancorfieldIf tools.build can completely replace depstar, I'll be happy to switch to it. That means: build JAR and uberJAR, do AOT, specify the main namespace, Multi-Release JAR support, merging data readers (and several other types of files) in appropriate ways... If it also includes Java compilation, I might change how we build/deploy a couple of OSS Java projects that we currently build from source (manually, as needed) and use local JAR files for.#2020-05-0506:56murtaza52there was a strange bug, I had deleted my .m2 directory some time back, and when I started my nrepl, it would start, but then on compiling a file it would give an error of a missing dep.
I changed the deps.edn file, and this triggered something, and on next nrepl startup, it downloaded the dependency. have never seen such a behaviour before.#2020-05-0513:25alexmillerthat all is what I would expect#2020-05-0513:26alexmillerclasspath is computed and then in cached in your .cpcache directory, with pointers into your .m2#2020-05-0513:27alexmillerclj script doesn't re-check that those files still exist (it could do so, but there are some performance implications and it's a little messy)#2020-05-0513:28alexmillerchanging the deps.edn makes the cached cp stale (it checks that the cp file is newer than all the source deps.edn files)#2020-05-0513:29alexmillerif you get into this situation again, use -Sforce to force the cp to be recomputed, which will also trigger a redownload#2020-05-0517:35rogerio haruo adachiHey, I'm running into some issues here, regarding transitive dependencies with deps with logging dependencies (cambium + unilog).
Scenario is as follows:
1. I have a shared library that has common libs ( datomic in memory, datomic client, kafka, integrant ... and logging)
2. An application that imports the above shared library
I have made all the necessary exclusions in item 1.
But when I import in item 2, I got conflicts between slf4j many libraries that I have excluded in item 1.
How can I maintain only dependencies required (with exclusions) in item 1 in a transitive way?#2020-05-0518:21alexmillercan you share your dep.edn to repro?#2020-05-0519:02rogerio haruo adachiSure.#2020-05-0519:04rogerio haruo adachithis is my libs deps
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
fundingcircle/jackdaw {:mvn/version "0.6.9"
:exclusions [org.slf4j/slf4j-log4j12
commons-logging/commons-logging
log4j/log4j
org.slf4j/slf4j-nop
org.slf4j/log4j-over-slf4j
org.clojure/tools.logging]}
datascript/datascript {:mvn/version "0.17.1"}
willa {:mvn/version "0.1.2"}
com.cognitect/anomalies {:mvn/version "0.1.12"}
com.gfredericks/test.chuck {:mvn/version "0.2.10"}
org.apache.kafka/kafka-streams {:mvn/version "2.3.0"
:exclusions [org.slf4j/slf4j-api]}
org.apache.kafka/kafka-streams-test-utils {:mvn/version "2.3.0"}
io.pedestal/pedestal.service {:mvn/version "0.5.7"}
io.pedestal/pedestal.route {:mvn/version "0.5.7"}
io.pedestal/pedestal.jetty {:mvn/version "0.5.7"}
integrant {:mvn/version "0.7.0"}
integrant/repl {:mvn/version "0.3.1"}
duct/core {:mvn/version "0.8.0"}
com.datomic/client-cloud {:mvn/version "0.8.78"
:exclusions [commons-logging/commons-logging]}
com.cognitect.aws/api {:mvn/version "0.8.408"
:exclusions [org.clojure/tools.logging]}
com.cognitect.aws/endpoints {:mvn/version "1.1.11.689"}
com.cognitect.aws/s3 {:mvn/version "780.2.583.0"}
rewrite-clj {:mvn/version "0.6.1"}
compute/datomic-client-memdb
{:git/url ""
:sha "fd880e70b45a0837f1604aa1230047cb74ec536e"
:exclusions [org.slf4j/slf4j-nop org.slf4j/log4j-over-slf4j org.slf4j/jcl-over-slf4j]}
badigeon/badigeon
{:git/url ""
:sha "dca97f9680a6ea204a2504c4414cafc4ba182a83"
:exclusions [org.slf4j/slf4j-nop
org.slf4j/jcl-over-slf4j]}
org.clojure/tools.cli {:mvn/version "0.4.2"}
cambium/cambium.core {:mvn/version "0.9.3"}
cambium/cambium.codec-simple {:mvn/version "0.9.3"}
spootnik/unilog {:mvn/version "0.7.24"}}#2020-05-0519:06rogerio haruo adachiAnd this is my app Deps
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
my.lib {:mvn/version "0.1.11-SNAPSHOT"} ;; This is where I import my lib and get conflicts over slf4j log4j libs
tick {:mvn/version "0.4.20-alpha"}
integrant/repl {:mvn/version "0.3.1"}
com.gfredericks/test.chuck {:mvn/version "0.2.10"}}#2020-05-0519:56alexmillerare you sure you're actually getting the latest version of your snapshot lib? by default, maven will only refresh snapshots once per day#2020-05-0522:37rogerio haruo adachiYes, I have been working with the latest version of my lib (snapshot)#2020-05-0520:13alexmillerwhat conflicts are you seeing?#2020-05-0520:14alexmillerI see log4j-over-slf4j under spootnik/unilog and other slf4j stuff under cambium - you should look at clj -Stree and see if what you're getting makes sense#2020-05-0520:16alexmillerif you want a more fine-grained analysis, use clj -Strace - that will generate a trace.edn file and you can walk through the :log to see each library it considers in order, whether it included it or not, and why#2020-05-0520:19alexmilleras a fallback, including it as an explicit dep in your app deps will take precedence over anything below (none of those exclusions are necessary then)#2020-05-0522:38rogerio haruo adachiThanks @alexmiller the fallback choice might work for now. Mean while I have extracted logging dependencies into a separate library to avoid conflicts.
Thanks for your help#2020-05-0615:16ikitommiAfter removing a directory from my .m2/repository, my clj failed the find dependencies using clj -m nrepl.cmdline -i -C. After doing any modification to the deps.edn , it reloaded the missing libs from clojars. Is there some local cache that caused this?#2020-05-0615:17ikitommie.g. “`deps.edn` unchanged, we have all the deps already”?#2020-05-0615:20delaguardoclass path cache is in .cpcache
and yes, unchanged deps.edn means all dependencies already downloaded#2020-05-0615:22mgrbyteThis came up yesterday too:
https://clojurians.slack.com/archives/C6QH853H8/p1588685140446600
also documented here:
https://clojure.org/reference/deps_and_cli#_classpath_caching#2020-05-0615:47ikitommiThanks, I should have read the history better.#2020-05-0616:49seancorfieldclj -Sforce is your friend here. I've tripped over this too.#2020-05-0619:57kennyWe have been getting this exception fairly frequently on CI builds. Retrying the job succeeds. Any idea what this is?
Error building classpath. class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1882)
at java.base/java.util.HashMap$TreeNode.treeify(HashMap.java:1998)
at java.base/java.util.HashMap.treeifyBin(HashMap.java:767)
at java.base/java.util.HashMap.putVal(HashMap.java:639)
at java.base/java.util.HashMap.put(HashMap.java:607)
at java.base/java.util.HashSet.add(HashSet.java:220)
at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:847)
at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependency(DefaultModelValidator.java:659)
at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependencies(DefaultModelValidator.java:583)
at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveModel(DefaultModelValidator.java:373)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:494)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
at org.apache.maven.model.building.DefaultModelBuilder.importDependencyManagement(DefaultModelBuilder.java:1299)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:474)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:292)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
at clojure.tools.deps.alpha.extensions.maven$eval813$fn__815.invoke(maven.clj:99)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at clojure.tools.deps.alpha$expand_deps$fn__1230$fn__1231.invoke(alpha.clj:214)
at clojure.tools.deps.alpha.util.concurrent$submit_task$task__500.invoke(concurrent.clj:33)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
#2020-05-0620:01hiredmanthat is a race condition when using a hash map from multiple threads, there was a recent-ish change to clj to do some stuff in parallel, and I bet if you disabled that those exceptions would go away#2020-05-0620:04hiredmanyou can stick :threads 1 in your deps.edn somewhere to download single threaded#2020-05-0620:11alexmillerNo it’s a clj flag #2020-05-0620:12alexmiller-Sthreads 1#2020-05-0620:12alexmiller(But going to be a resolve-args alias)#2020-05-0620:13alexmillerI haven’t seen this error anywhere before#2020-05-0620:16alexmillerI will file a ticket to take a closer look#2020-05-0916:31dpsuttoni'm trying to get a repl up and running in the tools.deps repo and running into a class not found exception for clojure.tools.deps.alpha.util.S3TransporterFactory. This seems quite reasonable but I don't see any instructions on how to build the java portion of this project. Does anyone know what incantation i need to provide to have this class on the classpath?#2020-05-0917:20alexmillerthe project is built with mvn package#2020-05-0917:26seancorfieldAll Contrib libraries are, yes?#2020-05-0917:26alexmilleryes#2020-05-0917:29alexmilleryou can use mvn to get a repl too with mvn clojure:repl but it's a pretty crappy repl#2020-05-0917:30alexmillerI typically use it as a maven project in Cursive#2020-05-0917:30dpsuttonah. i'm learning cursive and reflexively imported it as a deps project. i'll figure out how to switch that#2020-05-0917:31dpsuttonthanks much#2020-05-1211:47plexuswhen you have a dependency with a :local/root to another clojure-cli/deps.edn project, is there any way to have it pick up aliases in the deps.edn that is being referenced?#2020-05-1211:48plexusI thought they might "inherit" whatever aliases are in use, but seems that's not the case.#2020-05-1212:25alexmillerCurrently, no#2020-05-1212:27alexmillerWhat’s the actual problem you’re trying to solve?#2020-05-1407:44plexusSorry for the delay, I've elaborated on my use case on Ask Clojure: https://ask.clojure.org/index.php/9302/allow-transitive-deps-edn-dependencies-to-inherit-aliases#2020-05-1212:28vlaaadoh, oh, there is a relevant issue for that: https://clojure.atlassian.net/browse/TDEPS-116#2020-05-1216:12holyjakWhere does deps download dependencies to (ie which folder to back up for caching on CI)? ~/.m2?#2020-05-1216:15ghadi.m2 for maven, .gitlibs for git dependencies#2020-05-1216:43alexmiller#2020-05-1217:12ghadiin Github Actions I use:
- name: Cache Maven
uses: actions/#2020-05-1217:12ghadito handle caching#2020-05-1217:53avi👋 I feel like this might be a FAQ but I did search the mirror in Zulip to no avail, so:
I have a new project that a teammate is trying to work on. (They’re new/rusty to/with Clojure.)
All the Git deps in the deps.edn are specified with https URLs.
For some reason I don’t understand, when my teammate tries to start a REPL, and thereby triggers the retrieval of the deps, they’re getting this error:
Error building classpath. @github.com/cognitect-labs/anomalies: invalid privatekey: [
This is confusing because this dep has this coord:
{:git/url ""
:sha "2decbbb3ffd8e919c67d29f39ec6d920575f65c3"
:tag "0.1.12"
:deps/manifest :deps}
Full deps.edn: https://github.com/FundingCircle/md2c8e/blob/master/deps.edn
Is this a known problem with a known solution? Am I just missing something?
Thanks!#2020-05-1218:03alexmillerthat's weird looking#2020-05-1218:04alexmillerI cloned that repo and was able to start a repl#2020-05-1218:04aviYeah, I’ve been wracking my brain to try to think of something I did wrong here… maybe my teammate’s git config/environment is all wonky?#2020-05-1218:05alexmillerwell, deps mostly doesn't use your git config at all#2020-05-1218:05alexmillerI would definitely re-try with -Sforce just to make sure you don't have a bad cached classpath#2020-05-1218:06alexmillerand look at ~/.m2/settings.xml to see if there are any funky repos set up#2020-05-1218:06alexmillerand I guess you should look at ~/.clojure/deps.edn in case there is anything in there#2020-05-1218:07alexmillerit may be that the error is coming from the guts of jsch under jgit and the url has been translated so that being different than what's in deps.edn is not the issue#2020-05-1218:09aviok, thank you!#2020-05-1218:29alexmillermight be something weird seen by the ssh-agent that's totally unrelated, dunno#2020-05-1220:38johanatanhi, is it possible to have a deps.edn alias to shell out to normal bash commands ?#2020-05-1220:39johanatanwe have a preprocessing step that needs to run before figwheel-main is started (for example)#2020-05-1220:39johanatanand i'd like to do: clj -A:preprocess:fig:repl or clj -A:preprocess:fig:build#2020-05-1220:39alexmillerYes, depending on how much pain you’re willing to endure#2020-05-1220:39johanatanhehe, what are my poisons?#2020-05-1220:40alexmillerOr maybe not as a single command#2020-05-1220:40alexmillerA single clj command line will run one Clojure program#2020-05-1220:40alexmillerClojure can shell our and run anything#2020-05-1220:41johanatanyea, sounds fine. so you're saying to use -m the-main and shell out from the-main ?#2020-05-1220:42alexmillerYeah - either using clojure.shell or actually I find doing the interop with Java’s ProcessBuilder to be pretty trivial#2020-05-1220:42alexmillerBut if you’re running two commands, why not just write bash or Babashka or something #2020-05-1220:42johanatanit's just one command really: npm run build to build our webpack#2020-05-1220:54johanatanthis did the trick:
(ns my-ns.webpack
(:require
[clojure.java.shell]))
(defn -main []
(let [{:keys [exit out err]}
(clojure.java.shell/sh "npm" "run" "build" :dir "./webpack")]
(assert (= exit 0))
(println out)
(System/exit 0)))
#2020-05-1220:59ghadiwhat's the point#2020-05-1221:00ghadi@johanatan what do you gain doing this in clojure?#2020-05-1221:02johanatani gain the ability to do: clj -A:preprocess:fig:repl or clj -A:preprocess:fig:build . it's a single command for our engineers to enter rather than multiple . easier to document/ transfer. and it still allows clj to be the top-level entry point rather than a bash or other script which would have to be maintained if our deps.edn setup changes. this way, if deps.edn setup changes, it's merely the addition of another "-A"#2020-05-1221:02johanatanit's a "turnkey" solution. one command startup type situation#2020-05-1221:05johanatanalthough there seems to be a catch, the repl 's :main-opts is apparently overriding the preprocess one (which effectively turns preprocess into a no-op).#2020-05-1221:09alexmilleryes, that's my point above about an invocation of clj running only one program - you need to do two things here#2020-05-1221:05borkdude@johanatan Btw, deps.clj has -Scommand which may also do what you want here: https://github.com/borkdude/deps.clj#-scommand#2020-05-1221:06johanatan@borkdude yea, i saw that and explored it. but it requires two additional installs (at least): deps.clj and babashka.#2020-05-1221:06johanatanand since this will be running in GitHub Actions, i wanted to minimize installs#2020-05-1221:06borkdude@johanatan deps.clj doesn't require babashka, it's also available as a standalone executable#2020-05-1221:06borkdudebut point taken#2020-05-1221:07johanatanit would be nice if there could be a self-install of these tools#2020-05-1221:07johanatanlike say cljfmt-runner.check#2020-05-1221:07johanatandirectly within the deps.edn#2020-05-1221:07johanatanlike this i mean:
:lint {:extra-deps {com.jameslaverack/cljfmt-runner
{:git/url ""
:sha "97960e9a6464935534b5a6bab529e063d0027128"}}
:main-opts ["-m" "cljfmt-runner.check"]}
#2020-05-1221:08borkdude@johanatan deps.clj is available as a standalone executable. in CI you would only have to install that instead of clojure, so one thing instead of another thing, not an extra thing#2020-05-1221:08borkdudeand "installing" is just downloading that executable to the path, that's it#2020-05-1221:10johanatanyea but there is a github action for tools.deps:
uses: vouch-opensource/
unfortunately only one uses per step. but yea i could do a manual install#2020-05-1221:11borkdudejust giving you the option, I'm not saying you should 🙂 - running npm using a clojure command, feels like a detour to me#2020-05-1221:11johanatanhehe, true. i'll think about it. thanks!#2020-05-1221:12johanatando you think something like this could be possible in future?:
https://clojurians.slack.com/archives/C6QH853H8/p1589317666016100#2020-05-1221:12johanatan(for those tools)#2020-05-1221:12borkdudenpm also supports invoking commands. maybe the other way around would be more natural. see https://github.com/dundalek/closh/blob/master/package.json#L26-L56 for an example of this#2020-05-1221:13borkdudethat link doesn't work for me#2020-05-1221:13johanatan:lint {:extra-deps {com.jameslaverack/cljfmt-runner
{:git/url ""
:sha "97960e9a6464935534b5a6bab529e063d0027128"}}
:main-opts ["-m" "cljfmt-runner.check"]}
#2020-05-1221:13johanatan^^ including babashka as a git-based "extra-dep". i guess it would be impossible for deps.clj to be used that way#2020-05-1221:15borkdudedeps.clj doesn't install anything, it's a drop-in replacement for clojure that has some benefits for some people (like Windows users that want to invoke it in cmd.exe). And some custom options. Nothing more than that.#2020-05-1221:16borkdudeof course you can script an installer in some Clojure file and put that behind a main-opts#2020-05-1221:16johanatanah right. good point i guess it wouldn't require babashka but for this simple script, babashka would perhaps be a better option than fullblown clojure#2020-05-1221:17johanatanor.. i guess you're saying no need for either since i could invoke npm directly#2020-05-1221:17borkdudeyeah#2020-05-1221:17borkdudeit has a nice scripts option: https://github.com/dundalek/closh/blob/master/package.json#L26-L56#2020-05-1221:18johanatanah i see#2020-05-1221:20johanatanalright you've convinced me. deps.clj it is#2020-05-1221:20johanatandevelopers will just install it instead of clojure#2020-05-1221:20johanatan(or both if they so like)#2020-05-1221:22johanatanbut can the argument to -S be placed inside deps.edn ?#2020-05-1221:23johanatan(as an alias)#2020-05-1221:23borkdudeno, deps.clj doesn't change anything to deps.edn#2020-05-1221:24johanatanhmm, yea, not sure that helps in this case as it would mean two separate invocations of deps.clj#2020-05-1221:25johanatanprobably the "script" feature of package.json itself is the best bet since those can be strung together / combined#2020-05-1221:25borkdudeyeah, if you're already using npm I'd use that#2020-05-1221:25johanatan:+1:#2020-05-1221:33ghadivouch-opensource/tools.deps-build
I do not recommend that @johanatan. I have found a lot of success with DeLaGuardo/ in Github actions#2020-05-1221:34ghadiit allows you to run clj as many times as you need#2020-05-1221:34ghadivouch-opensource/tools.deps-build makes you cram it all into one invocation#2020-05-1221:39ghadiI added it to the wiki https://github.com/clojure/tools.deps.alpha/wiki/Tools#2020-05-1221:43johanatanOh yes, haha, that cram all into one was part of the motivation here as well. I’ll look at making the change to setup-clojure. Thx!#2020-05-1221:44ghadi- name: Setup Java
uses: actions/#2020-05-1221:44ghadithen I go buck wild#2020-05-1221:46johanatan:+1::skin-tone-2:#2020-05-1221:46johanatanIf you needed NPM, you’d just install it in a prior step?#2020-05-1221:46ghadinpm is installed by default in Actions, IIRC#2020-05-1221:47ghadialready available#2020-05-1221:47johanatanAh cool. Yea I thought so too #2020-05-1400:18fabraohello all, is there any way to exclude dependency of a lib in deps.edn ?#2020-05-1400:20alexmilleryou can use :exclusions [lib/name] in a coordinate#2020-05-1400:21alexmillerI'm assuming you're talking about a transitive dependency#2020-05-1400:21alexmillerthere is no global exclusions right now#2020-05-1400:22fabraoHello Alex, I´m doing this
{:paths ["src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
mount {:mvn/version "0.1.16"}
com.sikulix/sikulixapi {:mvn/version "2.0.4"
:exclusions [org.bouncycastle/bctsp-jdk14]}}}
and still getting
clj -J'-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl}'
Error building classpath. Could not find artifact bouncycastle:bctsp-jdk14:jar:138 in central ()#2020-05-1400:23fabraowhat is maybe wrong?#2020-05-1400:24alexmillernot sure, that seems like good deps syntax#2020-05-1400:25alexmillerI can repro locally#2020-05-1400:25alexmillerseems like its in maven central to me https://repo1.maven.org/maven2/org/bouncycastle/bctsp-jdk14/#2020-05-1400:26alexmillerthat 138 looks suspicious to me#2020-05-1400:26alexmillermaybe should be 1.38#2020-05-1400:26fabraoso, excluding is not an option for this?#2020-05-1400:26alexmillerwhich is an actual version - I assume that's in com.sikulix/sikulixapi deps#2020-05-1400:27alexmillermight be an edge case where it's failing while getting the metadata for something that it's going to exclude#2020-05-1400:27alexmillerI am getting on a call, but I can look at it tomorrow#2020-05-1400:27fabraooh, ok, thanks#2020-05-1400:28fabraoI´ll try with lein#2020-05-1400:31seancorfield@fabrao apparently what you want to exclude is bouncycastle/bctsp-jdk14 (without the org.)#2020-05-1400:32seancorfieldI noticed in the initial set of download messages:
Downloading: org/jboss/logging/jboss-logging/3.1.4.GA/jboss-logging-3.1.4.GA.pom from central
Downloading: org/jboss/jboss-parent/9/jboss-parent-9.pom from central
Downloading: bouncycastle/bctsp-jdk14/138/bctsp-jdk14-138.pom from central <======== LOOK AT THIS LINE!
Downloading: org/bouncycastle/bctsp-jdk14/1.38/bctsp-jdk14-1.38.pom from central
Downloading: org/ghost4j/ghost4j/1.0.1/ghost4j-1.0.1.jar from central
#2020-05-1400:32fabrao@seancorfield You are the BOSS !!! That´s it !!! Thank you#2020-05-1400:32seancorfieldIt's this: https://search.maven.org/artifact/bouncycastle/bctsp-jdk14#2020-05-1400:33seancorfield(which looks ancient and maybe even bogus?)#2020-05-1400:55alexmilleryeah, that must be old or something#2020-05-1400:59alexmillerin case you're wondering the path to badness is
com.sikulix/sikulixapi 2.0.4
net.sourceforge.tess4j/tess4j 4.4.1
org.ghost4j/ghost4j 1.0.1
com.lowagie/itext 2.1.7
bouncycastle/bcprov-jdk14 138#2020-05-1400:59alexmillerthere are newer versions of stuff lower down, didn't investigate in detail but probably could update some stuff to newer there to fix#2020-05-1401:17fabraoI could use with excludes like Sean said, thanks#2020-05-1407:37Andreas EdvardssonI have added a plain java class alongside my clojure code. Is it possible to have it compiled as a part of the ordinary clojure compilation? In lein there seems to be a :java-source-paths option, is there something corresponding in tools.deps?#2020-05-1419:32just.sultanovI use this solution. If you want you can use something like this:
https://gist.github.com/just-sultanov/e2f61734e22e2ed3981216ed9b880bc0#2020-05-1407:46plexusnot in tools.deps. Some of the build tools based on tools.deps can do it though. Or you may be able to get away with compiling them manually, e.g.
for x in **/*.java; do javac -cp `clojure -Spath` $x ; done
#2020-05-1408:35Andreas EdvardssonThanks for the answer! The class is expected to be more or less static, so just a manual javac invocation indeed seems to be the simplest solution. :)#2020-05-1422:04bringeHi, I'm a bit new to the clojure command line tools. I'm wondering if there is an equivalent to lein deps and this is because I'm using it with docker. It is recommended with docker to download dependencies in a separate step so that they aren't re-downloaded if none changed, such as:
FROM clojure
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY project.clj /usr/src/app/
RUN lein deps
COPY . /usr/src/app
RUN mv "$(lein uberjar | sed -n 's/^Created \(.*standalone\.jar\)/\1/p')" app-standalone.jar
CMD ["java", "-jar", "app-standalone.jar"]
#2020-05-1422:04alexmilleryou can do clj -e nil to trigger deps to resolve and download#2020-05-1422:04bringeAhhh thanks!#2020-05-1422:11pmonksYeah I had a need for this recently (in a TravisCI build), and ended up doing clojure -Srepro -e '(println "Dependencies downloaded")'. Felt a bit hackish, but glad to see that’s basically how it’s done.#2020-05-1422:12pmonksI stand corrected - it was a GitHub action/workflow thingy - but basically the same idea.#2020-05-1615:01gfrederickscan a deps.edn provide jvm-opts for default calls, without having to specify an alias?#2020-05-1615:01alexmillerNo#2020-05-1615:02gfredericksis that intentional, or a maybe-future-feature?#2020-05-1615:02alexmillerMaybe future #2020-05-1615:02alexmillerBut it seems weird that all clj invocations in a project would share those #2020-05-1615:03alexmillerWhich is why I haven’t added it yet#2020-05-1615:03gfrederickswhen imagining it was intentional I started thinking that maybe for a nontrivial project clojure isn't meant to be called directly, and you'd setup bin/ scripts that call clojure with appropriate aliases#2020-05-1615:04gfredericksat least for running specific code; maybe it's fine for launching a repl#2020-05-1615:05alexmillerLike maybe it would make more sense if it was scoped only to running a repl#2020-05-1615:05alexmillerOr running a specific main #2020-05-1615:06gfredericksyeah, specific main would definitely be adequate for anything I want to do#2020-05-1615:12alexmillerBut if you’re doing that, don’t you already have an alias where you can put it?#2020-05-1615:13alexmillerOr is it that you don’t want to repeat it in aliases#2020-05-1615:17gfrederickshmm, you're right, I guess a human typing in a main ns might not be very common
so my particular situation, which is maybe not common enough to be worth targeting, is that my "project" mostly consists of a monolithic clj file at the root, and I just run clojure code.clj a lot
but I'd like to have it with a restricted -Xmx by default, and can't do that; I just have to remember to run clojure -Afoo code.clj instead#2020-05-1615:17gfredericksI think "stop being weird" is a perfectly good answer here, I was just curious#2020-05-1619:11seancorfieldOr even just clojure -A:foo since you could add code.clj to the alias (as a :main-opts element), right?#2020-05-1619:16seancorfield{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "324157535c517276776179667d621f6367006778037c"}, :content ("[email protected]")}#2020-05-1621:30dominicmIf tools.deps exposed a :init-opts option in deps.edn, that would remove the need for multiple mains. Most uses (I've had) involve side-effects like launching an nrepl for personal dev purposes. If I use a main for that, I can't use a project-specific main.#2020-05-1621:56seancorfield:main-opts can contain -e and -i -- and you can combine multiple :main-opts from multiple aliases @dominicm#2020-05-1621:56seancorfieldThat's partly why I separate out :test and :runner for example.#2020-05-1621:58seancorfieldI admit, I do miss the ability of combining aliases in other aliases -- that's probably the thing that would add the most ergonomic utility for me. That's really why we have a shell script wrapped around clojure: to do some "alias expansion"#2020-05-1621:58dominicm@seancorfield you can only have one :main-opts active from an alias at a time.#2020-05-1621:59dominicmCompare with jvm-opts which does concatenate.#2020-05-1622:00seancorfieldUgh! Yeah, you're right. I'd convinced myself they combine...#2020-05-1622:06seancorfieldWhich means I change I made recently to my dot-clojure file doesn't actually work.#2020-05-1622:09seancorfield(well, it does work, but not quite how I thought and it explains why I still need a manual step at the end)#2020-05-1622:10seancorfieldSo, yeah, you've now convinced me that having an :init-opts for just "init opts" that actually combined like :jvm-opts would be a good thing 🙂#2020-05-1622:10seancorfield(or, frankly, even if :main-opts itself combined and it was just "caveat programmer" to use it that way)#2020-05-1622:14dominicmThe problem with main opts composing is that there's an order problem. Not so with init-opts.
I suppose people may also be relying on the fact that main-opts conflict.#2020-05-1622:15seancorfieldIf you have multi -e options you might want them to execute in a particular order 🙂#2020-05-1622:16seancorfieldI added -e to my :deps alias to set up a DCL, so that if you do -A:deps:socket then you get a DCL in place before the Socket REPL is created so you able to use add-lib via Atom.#2020-05-1622:16seancorfieldBut if you do -A:deps:socket:rebl-11 you get a REBL window without DCL 😞#2020-05-1622:18seancorfieldSo you have to do -A:deps:socket -R:rebl-11 -r (omitting the :main-opts from REBL) and that sets up a DCL and starts a Socket REPL and finally starts a command-line REPL... and in there you can do ((requiring-resolve 'cognitect.rebl/ui)) to start the REBL window and then it does have a DCL.#2020-05-1622:18seancorfield(this is all a workaround because Socket REPLs don't use DCLs by default)#2020-05-1622:23dominicm@seancorfield my implementation of this in user space might interest you. You set properties using jvm opts, then have a main which reads those properties and runs evaluations from them.#2020-05-1623:41alexmillerEarly versions of clj did this :)#2020-05-1622:23dominicmImplementation is a little strong perhaps. Idea :).#2020-05-1623:01alexmillerRe init opts, at some point you’re programming, just write a clj and call it#2020-05-1623:46seancorfieldWell, that was why, in the end, we wrote our little dev/repl.clj script at work -- it looks at what's on the classpath and starts a Socket REPL, an nREPL Server, REBL, or a regular REPL.#2020-05-1700:19alexmillerThere are a couple generic ideas we have in this area that we will be working on, not sure yet if they are clj or clojure.main#2020-05-1706:37dominicmThe composition is important I think. I don't want to maintain a clj for every project I work on.#2020-05-1708:22didibusI find writing a little clj file for custom stuff pretty convenient. I wouldn't say no to more systematic approaches if clj adds them. Just saying, it's not that bad to have a .clj file#2020-05-1708:23didibusIt means everyone who knows Clojure can understand the build script, since it's just a clj file. As opposed to having to learn some new configuration system.#2020-05-1708:24didibusThe only downside is paying the startup price on every script execution.#2020-05-1709:12dominicmDifferent contexts I think. A project can have a single clj file, but a user might have many they compose in different ways.#2020-05-1709:12dominicmFor example, in one project I might want to run a clj file which launches postgres in the background if it's not already started, but I want that to work alongside the project's clj file.
clj -i pg.clj -i theproject.clj
#2020-05-1713:05alexmillerclj is for running a (single) Clojure program. It sounds like you’re trying to run multiple programs. At that point, use bash or babashka or write a generic clj tool that launches multiple programs. #2020-05-1713:44dominicmI want to have side effects on the world before a program runs. Usually within the same jvm.#2020-05-1717:26wilkerluciohello, I'm having some version conflicts problem in my project, and on trying to debug I remember on lein when we ask for the tree it suggests some exclusions to avoid dependency conflicts, is there any such tool like this for use with deps.edn dependencies?#2020-05-1717:31alexmillerdepot can help with some similar things#2020-05-1718:42dominicmTo just clarify what I was saying above ^ I'm mostly interested in dev tooling. Things like scope-capture, nrepl, etc. So I want to effect the running JVM.#2020-05-2006:15dharriganis it necessary to include clojure as a dep in the deps.edn? I've seen a few projects with it included.#2020-05-2010:34dominicmThere was a news-and-articles by daniel janus recently about this#2020-05-2011:22dharriganooh. ta.#2020-05-2012:43alexmillerYes, but it is included by default#2020-05-2012:48alexmillerSo I guess no :)#2020-05-2012:49dominicmMy personal thoughts are that you should mark your minimum required version. Because if you don't update your computer (for some reason) and then rock up on a project using Clojure 1.22 features, and you've got 1.15 you'll be scratching your head for a while. One less thing to manage externally.#2020-05-2013:47dharriganIt's more to do with docker containers. If I can remove the dependency (as it's already there as part of openjdk-11-tools-deps-slim-buster), then I can remove it from the deps.edn file, thus making the binary ever so slightly smaller.#2020-05-2013:56dominicmHmm, I think you'll only get one copy either way...#2020-05-2014:01alexmilleryou need to have it, and you will via the install deps.edn, it's just a question of version#2020-05-2014:01dominicmMaybe by removing it, you're just forcing to match the version in your docker.#2020-05-2016:12dharriganProbably safer to leave it in.#2020-05-2016:12dharriganAfter all, storage is cheap 🙂#2020-05-2007:41PiersHello, I've seen examples of installing dependencies using clj -Sdeps '{:deps {bidi {:mvn/version "2.1.6"}}}' but I was wondering if it's possible to instruct clj to install dependencies listed in deps.edn without doing anything else? Similar to lein deps I suppose.#2020-05-2007:52delaguardoclj -Spath as an option. it will download dependencies and print out calculated class-path#2020-05-2011:38seancorfieldI think I've seen clj -e nil given as a quieter example. #2020-05-2012:24teodorluI've used clj -e :deps-download-complete when building Docker images#2020-05-2021:45PiersThanks for the suggestions, that's really helpful. My use case is also for building Docker images.#2020-05-2017:42dvingois there a way via clj to see all available aliases?#2020-05-2017:46alexmillerNot currently. I’ve been thinking about adding thAt#2020-05-2017:50dvingonp, thanks#2020-05-2018:46aviI apologize for the probably FAQ, but I have searched for this… when I specify a gitlib as a dependency, is there a way to specify the path within the gitlib to the source code? e.g. there’s a dep I’d like to use which has its Clojure code in src/clj/#2020-05-2019:07dominicm@aviflax 👋 That would be :deps/root :)#2020-05-2019:07dominicmhttps://github.com/roman01la/uix see an example in UIx#2020-05-2019:10avi👋 Hi Dominic, thanks!
I saw :deps/root in the docs, but the description therein says:
> Specifies the relative path within the root to search for the manifest file.
In the case of this particular dep, the manifest file doesn’t even really exist… I’m faking it by manually setting :deps/manifest to :deps, to finagle tools.deps into treating this dep as if it has an empty deps.edn, which works in this case because the dep has no dependencies itself.
Anyway, that’s why I didn’t try it, but it makes sense that it would also change how the lib is added to the classpath. I’ll try it. Thanks!#2020-05-2019:12aviHmm that’s not quite working. I set :deps/root to src/clj but the path added to the classpath is /src/clj/src 😞#2020-05-2019:15dominicmOh I see "to the source code". Yeah, that makes sense. I'm not sure you can get away with that.#2020-05-2019:16aviI figured. I kinda wish there was a way to override that src segment that tools.deps adds to the end of the root. Something like :deps/code-path or something.#2020-05-2019:16dominicmhttps://github.com/clojure/tools.deps.alpha/blob/f94815dd55bdf5eb30ac8fa075c39e757cbbcca5/src/main/clojure/clojure/tools/deps/alpha/extensions/deps.clj#L29-L34#2020-05-2019:16dominicmIt would probably be :deps/paths or something :).#2020-05-2019:18aviThanks for the link, but I’m not really understanding that function. I think you’re pointing out that there isn’t currently a way to do this?#2020-05-2019:18aviI was thinking maybe :deps/paths but I’m not sure about the plural part. I’m not sure it makes sense to add multiple entries to the classpath for a dep.#2020-05-2019:24hiredman:deps/root#2020-05-2019:24aviTried that already.#2020-05-2019:24aviDoesn’t work in this case, see above.#2020-05-2019:26dominicm@aviflax I am saying there's no way to do this currently :)#2020-05-2019:26aviYup, got it. Thanks though!#2020-05-2019:26aviI guess I’ll open a ticket/issue as a feature request, couldn’t hurt.#2020-05-2019:28alexmillerI think this already does what you want#2020-05-2019:29aviOh! What am I missing?#2020-05-2019:29alexmilleror rather let me back up...#2020-05-2019:29alexmillerwhat kind of project is the git dep?#2020-05-2019:29alexmillerwhat kind of manifest file (pom, lein, deps)#2020-05-2019:29aviproject.clj#2020-05-2019:29avibut it has no deps#2020-05-2019:29aviso I’m specifying :deps/manifest as :deps#2020-05-2019:30alexmillerthat should just work - what doesn't work?#2020-05-2019:30aviIn the dep’s dir structure, the Clojure code is in `src/clj/`#2020-05-2019:30alexmillerah, ok#2020-05-2019:31aviI tried setting `:deps/root` to `src/clj` but the path added to the classpath was then `/src/clj/src`#2020-05-2019:31alexmilleryeah, there is no way right now to externally set a git dep's paths#2020-05-2019:31aviah, ok#2020-05-2019:31aviwould a ticket be helpful?#2020-05-2019:31alexmillersure#2020-05-2019:31avi👌#2020-05-2019:31alexmilleryou coudl fork it and add a deps.edn though#2020-05-2019:32aviah, good point. Maybe I will. Thanks!#2020-05-2019:32alexmilleror clone and do the same with a local project#2020-05-2019:32aviI was thinking maybe I’ll open a PR to contribute the deps.edn to the main project; I’d rather not use a fork indefinitely#2020-05-2019:35alexmilleror that#2020-05-2019:52aviI posted the request as a question to ask as that seems to be the preferred way to suggest an issue these days: https://ask.clojure.org/index.php/9314/is-it-possible-to-override-the-paths-of-a-git-dep#2020-05-2019:54alexmillerI think the question presupposes the need to do this, but maybe the broader request is about supporting lein projects#2020-05-2019:54aviyeah#2020-05-2019:54aviagreed!#2020-05-2019:55alexmilleror even more broader, supporting extensible project types, one of which could be lein#2020-05-2019:55alexmillertools.deps is actually written to support this but clj does not expose that capability currently (b/c there are a lot of questions about how to do it well)#2020-05-2019:56aviI can see it being a kind of support nightmare#2020-05-2019:56aviI’m sure a million edge cases would pop up#2020-05-2019:57avimaybe once tools.deps drops the “alpha” — then there could be a subsequent alpha/beta/whatever release that would enable the feature, with the caveat that it’s unstable, only for testing, etc?#2020-05-2120:37dominicmSilly question, what does override-deps do that extra-deps doesn't? I realized I've been using extra-deps in my deps.edn when I want to override a dep.#2020-05-2120:46alexmilleroverride replaces, extra adds#2020-05-2120:46alexmilleroverride does not add#2020-05-2120:47alexmillerthat is, it is ignored if you don't already have the lib#2020-05-2120:49alexmillerI guess it would be more accurate to say override replaces and extra adds or replaces#2020-05-2120:51alexmillerfurther refining...
• extra adds new top-level deps (which may effectively replace existing top or transitive deps)
• override replaces the version of an existing top or transitive dep #2020-05-2120:52alexmillerin both cases, you should be guaranteed to get the version you've said - in extra, because it's at the top level, and in override because it replaces at all levels#2020-05-2121:29seancorfield@dominicm For a concrete use case, we use :override-deps heavily at work to "pin" versions of libraries across our entire repo. If a particular app doesn't use a given dependency, the :override-deps does nothing. For deps we expect to be overridden, we specify the coords as just {} (since they'll be supplied via the override). Does that help?#2020-05-2121:31seancorfieldIn a community context: https://github.com/seancorfield/dot-clojure/blob/master/deps.edn#L4-L17 -- those aliases let you pick a specific version of Clojure by overriding whatever would otherwise be pulled in.#2020-05-2121:35dominicmCool, thanks for both your input. 😁#2020-05-2612:49AudriusHi, I have
Error building classpath.
java.lang.NullPointerException
in one project when running what ever. I believe there could be a bug int the deps tool
Here is source: https://github.com/audriu/fake#2020-05-2612:55dpsuttonYour figwheel main has “mvn/version” as a symbol rather than a keyword#2020-05-2613:07alexmillerI’ll take a look at why you’re seeing that error message, I would expect better#2020-05-2613:08AudriusI am very happy to help at least by contributing this 😎#2020-05-2613:40dominicmFull stack:
java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
at java.util.regex.Matcher.reset(Matcher.java:309)
at java.util.regex.Matcher.<init>(Matcher.java:229)
at java.util.regex.Pattern.matcher(Pattern.java:1093)
at clojure.core$re_matcher.invokeStatic(core.clj:4856)
at clojure.core$re_find.invokeStatic(core.clj:4898)
at clojure.core$re_find.invoke(core.clj:4898)
at clojure.tools.deps.alpha.util.maven$version_range_QMARK_.invokeStatic(maven.clj:250)
at clojure.tools.deps.alpha.util.maven$version_range_QMARK_.invoke(maven.clj:248)
at clojure.tools.deps.alpha.extensions.maven$eval703$fn__706.invoke(maven.clj:46)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$canonicalize_deps$fn__1136.invoke(alpha.clj:69)
at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
at clojure.core.protocols$fn__8140.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__8140.invoke(protocols.clj:75)
at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6828)
at clojure.core$reduce.invoke(core.clj:6810)
at clojure.tools.deps.alpha$canonicalize_deps.invokeStatic(alpha.clj:68)
at clojure.tools.deps.alpha$canonicalize_deps.invoke(alpha.clj:66)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:236)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
#2020-05-2613:41alexmillerThx#2020-05-2613:41dominicmOh, I see the problem#2020-05-2613:41dominicmhttps://github.com/audriu/fake/blob/master/deps.edn#L4 it's a symbol, not a keyword. So there's no :version defined.#2020-05-2613:42dominicmI'm guessing the ns code doesn't care about keyword vs symbol though#2020-05-2613:43dominicmYeah, it gets through coord-type because of the way it's written. But then version can't be read at any point, because it's not a keyword.#2020-05-2613:44dominicmThere's also the fact it's figheel-main instead of fig*w*heel-main :)#2020-05-2717:05alexmilleradded some error checking for this for next release, now you'll see:
Error building classpath. No :mvn/version specified for com.bhauman/figheel-main
#2020-05-2613:44dominicm@stebokas also, you've typo'd figwheel-main :)
- com.bhauman/figheel-main {mvn/version "0.1.9"}
+ com.bhauman/figwheel-main {:mvn/version "0.1.9"}
#2020-05-2613:46bhaumanthe most common typo ever 🙄#2020-05-2713:17respatializedIs there a way using deps.edn to add code that gets eval'd on startup for an alias or for a repl launched using clj? The problem I'm specifically trying to solve is requiring jansi-clj for any terminal clojure session so it picks up my system terminal colors, but I was wondering if there was a generally recommended approach to the problem of "default startup code" using tools-deps.#2020-05-2713:18alexmilleryou can use clojure.main's -i or -e sometimes for things likee this#2020-05-2713:19alexmillerclj -h will tell you more#2020-05-2713:19alexmiller-i loads an arbitrary clj file, -e runs an expression#2020-05-2713:22respatializedgot it, so it's the simple matter of combining -e (require 'jansi-clj.auto) -r to launch a repl with some short startup code, and -i "/path/to/file" -r for anything longer. thanks!#2020-05-2713:23alexmilleryep, and you can put that in an alias with :main-opts#2020-05-2713:23alexmillerso clj -A:repl or whatever#2020-05-2915:17jjttjjI'm trying to use andare, a replacement for core async with the same namespace hierarchy, in a project that also depends on things which depend on core.async (shadow-cljs specifically). However just adding core.async to the :exclusions for shadow-cljs seems to prevent it's namespaces which use core.async from working. Is there something I'm missing?#2020-05-2915:21seancorfield@jjttjj Maybe shadow-cljs is AOT'd? That would be my guess. (your problem doesn't sound like it would be specific to Clojure CLI / deps.edn / tools.deps.alpha?)#2020-05-2915:24thhellerit is only AOT if you request the AOT variant#2020-05-2915:24thhellerI'm guessing that andare just isn't fully compatible? how does it fail?#2020-05-2915:28jjttjjFile: jar:file:C:\Users\justin\.m2\repository\thheller\shadow-client\1.3.2\shadow-client-1.3.2.jar!/shadow/util.cljs
failed to require macro-ns "shadow.util", it was required by "shadow.util"
Error in phase :compile-syntax-check
RuntimeException: No such var: ioc/state-machine
#2020-05-2915:31thhellerguess that doesn't exist in andare? it is an internal cljs.core var#2020-05-2915:31jjttjjYup, I see it doesn't now, thanks#2020-05-2915:32jjttjjIt looks like shadow.util in shadow-client uses some core.async internals#2020-05-2915:35thhellerit does. I can probably remove it but I'm not sure trying to replace the entire core.async impl is sustainable#2020-05-2915:35thhelleryou are doing this for self-hosted CLJS right?#2020-05-2915:37jjttjjyes. I wouldn't necessarily request that either, just messing around right now 🙂#2020-05-2915:39jjttjjalthough it looks possible that your go! macro that requires it might not be used much at all in shadow#2020-05-2915:43ghadioh look, an incompatible thing with the same name#2020-05-2915:45seancorfield^ That was my first thought too. Any "replacement" really should use at least a different root namespace, precisely to avoid these sorts of conflicts 😐#2020-05-2915:47dpsuttoni think its so self-host can use it. it needs to be named the same#2020-05-2915:47jjttjjmakes sense. I'll prob just fork andare and rename things#2020-05-2916:09dpsuttonrename to what?#2020-05-2916:09dpsutton> Andare preserves the namespaces present in core.async. Thus, bootstrap-compatible ClojureScript code that makes use of core.async can operate in self-hosted environments if you make the Andare artifact available for loading in lieu of the official core.async artifact.#2020-05-2916:11seancorfieldAlso, none of this seems relevant to this channel's topic -- am I missing something there @jjttjj?#2020-05-2916:13jjttjjNope, sorry about that. Thought it might be tools.deps specific at first (relating to exclusions) but realize now that's not the case#2020-05-2921:39FlavaDaveI’m trying to file a ticket at https://clojure.atlassian.net/projects/TDEPS/issues/?filter=allissues but it is saying I’m not authorized to create issues. What is the best way for me to report an issue?#2020-05-2921:39seancorfieldhttp://ask.clojure.org#2020-05-2921:39seancorfieldTag it with the tools.deps tag...#2020-05-2921:40seancorfieldhttps://ask.clojure.org/index.php/questions/contrib-libs/tools-deps-alpha are existing issues/questions.#2020-05-2921:40lvhah, I thought that was just for discussion, not for bug reports#2020-05-2921:41seancorfield@dgonsalves22 The About page explains why that exists https://ask.clojure.org/index.php/about and the official site's contributing page links to that forum https://clojure.org/community/contributing#_reporting_problems_and_requesting_enhancements /cc @lvh#2020-05-3109:39p-himikLet me know if this is not the right place for this question.
I'm trying to build an uberjar with depstar. It's my (almost) first attempt to do something like that.
Going by this snippet from the README:
# generate pom.xml (or create it manually)
clojure -Spom
# build the uberjar with AOT compilation
clojure -A:depstar -m hf.depstar.uberjar MyProject.jar -C -m project.core
# Main-Class: project.core
java -jar MyProject.jar
Did all that in a very simple "hello world" project.
Running that gives me
Error: Could not find or load main class project.core
Caused by: java.lang.ClassNotFoundException: project.core
I opened the jar with jd-gui and I see that there's indeed no project.core class even though the manifest mentions it.
Running a non-AOT variant with java -jar MyProject.jar -m project.core works just fine.
Are there some obvious steps that I might have missed? Or can it be an issue with depstar?#2020-05-3109:45seancorfieldIs your project on GitHub so I can take a look @p-himik?#2020-05-3109:46seancorfieldDoes your project.core namespace have (:gen-class) in the ns form? Does it have a -main function?#2020-05-3109:47seancorfield(if you create an app project with clj-new, it'll have all the right bits in it that you need for an uberjar -- it'll even have the :aliases already set up in deps.edn)#2020-05-3110:22p-himik(:gen-class) seems to be it, thanks! I had no idea it was needed for -main with AOT. I'll try clj-new as well.#2020-06-0105:18didibusWould it make sense to add a npm resolver?#2020-06-0105:20didibusRight now, it seems a ClojureScript lib can't depend on another ClojureScript lib that depends on a JS lib unless someone repackages the JS lib into Maven. But since tools.deps has custom resolvers, if it could resolve from npm, it seems it would allow this scenario to work#2020-06-0105:23seancorfieldWhen you say "Maven", does that include the cljsjs infrastructure or whatever it is?#2020-06-0105:25didibusYa it does#2020-06-0105:26didibuscljsjs just repackages NPM packages into Clojars#2020-06-0105:27didibusAs it does so, it might also go ahead and perform some minification on it, and it'll add the required extern files.#2020-06-0105:29didibusSo cljsjs is convenient when you're too lazy to minify or write extern files yourself. But right now, it is also mandatory just to be able to resolve deps and deps conflicts. At least this is my understanding.#2020-06-0105:32didibusI guess the challenge of a NPM resolver would be transitive NPM deps. Tools.deps would need to understand the npm package.json (the equivalent of pom.xml) and all that#2020-06-0105:36seancorfieldI would imagine the #clojurescript channel would be the best place to petition for such a thing? How does David Nolen feel about cljs dependencies from the NPM/JS world? I saw the recent changes to make it easier to publish cljs stuff to NPM (as I understood the news)...#2020-06-0105:49didibusHum, ya not sure. From my understanding, the new bundle target isn't really about publishing to NPM.
Basically, when you want to deploy a ClojureScript app, you need to do a sort of uberjar, but in JS world it's called a bundle.
The ClojureScript compiler has the capability to bundle your ClojureScript code and all its dependencies together. Since all users will need to download your bundle when viewing your site, bundle size is a sensitive topic.
ClojureScript code and ClojureScript dependencies as well as JS dependencies that are designed with Google Closure support can be bundled by the Google Closure compiler in a super optimized way (in terms of having a very small size). It renamed all vars to super short names, removed all whitespace, and does dead code elimination at the function and variable level.
That said, JS dependencies not designed for Google Closure can't go through it. So ClojureScript compiler lets you specify those as "foreign-libs". In which case, it will still bundle them up, but it won't optimize them at all.
For a project that depends on a lot of such library, that isn't very good, because it still results in a large bundle size.
There exist other bundlers for JS, which can take those type of libraries and bundle them with a reduced size. But until now, if you tried to used them to bundle the already bundled output of the ClojureScript compiler, it would fail.
The new bundle target for the ClojureScript compiler fixes this. It makes sure to output a bundle that can be further bundled by JS bundlers.
Nothing about it really has anything to do with package management. At least this is my current understanding.#2020-06-0117:17seancorfield@U0K064KQV Thanks for the explanation. Some of that I already knew but I was hazy on the relationship between cljs and bundles. One day I hope to find the time/bandwidth to dig back into cljs and maybe build something interesting so I can learn about the tooling available now (2014 was the last time I tried to build anything serious in cljs and a lot has changed since then! 🙂 )#2020-06-0121:45didibusMy pleasure, sometimes I write detailed explanations but it's really for myself that I do it. Forces me to put some order in my understanding and tests also whether I'm still missing some bits.#2020-06-0107:55thhellerFWIW I don't think it make sense to move this into deps.edn. I doubt Alex wants to maintain and interact with the insanity that is npm. We already have the mechanism to declare npm dependencies and installing them in CLJS directly, just a bit basic maybe. can always add more options to that when needed.#2020-06-0108:23didibusIs that through shadow-cljs? Or without?#2020-06-0108:28thhellerCLJS has --install-deps via cljs.main and shadow-cljs just does it automatically on startup#2020-06-0108:35didibusHum, so there's something I don't understand then. Install-deps is unaware of versions. And it does no transitive deps handling. Or I haven't figured out how#2020-06-0108:36didibusSo for example, if lib A wants to declare some npm-deps, and app X wants to use lib A, app X won't automatically pull down lib A's deps#2020-06-0108:39thhellerin CLJS automatically no, you have to run --install-deps in app X#2020-06-0108:39thhellerin shadow-cljs it is automatic#2020-06-0108:40didibusWell, I'll definitly be trying shadow-cljs eventually. But for now, --install-deps, even run manually doesn't pull transitive dependencies for me#2020-06-0108:40thhellerboth look for deps.cljs files in the libraries and use :npm-deps {"some-lib" "some-version"} to install#2020-06-0108:40didibusOh, what is a deps.cljs file 😛#2020-06-0108:40thhellerwell do this libs you are testing contain the proper deps.cljs? many that still just depend on cljsjs don't#2020-06-0108:41didibusI am writing the lib. First time I hear about a deps.cljs file#2020-06-0108:41thhellereg. https://github.com/reagent-project/reagent/blob/master/src/deps.cljs#2020-06-0108:43didibusOk, official doc seems here: https://clojurescript.org/reference/packaging-foreign-deps#2020-06-0108:45thhellerhmm that doesn't even mention npm-deps#2020-06-0108:45thhellermaybe there are no docs for this? I always thought there were#2020-06-0108:46didibusThere doesn't seem to be. And I felt like I had gone over most of the docs looking for something like this. Maybe it just needs some better doc#2020-06-0108:47didibusSo in shadow-cljs you just leverage the compiler install-deps feature under the hood? Or you make use of npm directly? Or you implemented a commonJS repository resolver of your own?#2020-06-0108:48thhellerno I don't use the CLJS install-deps ... mostly because it didn't exist when I wrote the feature for shadow-cljs#2020-06-0108:48thhellerI don't handle any package installing whatsoever, it just runs npm install nothing more.#2020-06-0108:48thheller(same as CLJS does nowadays for --install-deps)#2020-06-0108:49didibusOk, and you grab the-package and the-version from the deps.cljs file on the resource path?#2020-06-0108:54didibusJust trying to figure out how to best package my lib. So if I make use of a npm deps, I just declare it in the deps.cljs along with foreign-libs and externs. And now anyone would be able to just add a dependency to my lib and unless they have a newer version of the npm deps themselves, everything would work as is and they'd get my lib and my transitive npm deps (assuming shadow or needing to run install-deps manually otherwise)#2020-06-0108:56thhellergenerally it is either/or. either you depends on :npm-deps OR you use foreign-libs. don't mix them.#2020-06-0108:57didibusAnd where am I supposed to put the deps.cljs ? That weird doc mentions inside a src folder? But I assume just anywhere on a resource path of my deps.edn? Or should it be project root?#2020-06-0108:57thhellerthe file needs to be in the published .jar file at the root#2020-06-0108:58thhellerso generally any source-path will do#2020-06-0108:58didibusOk, and if I'm temporarily using a local dep? Same logic applies?#2020-06-0108:59thhellerfor CLJS don't know. for shadow-cljs yes, it just loads any deps.cljs files on the classpath#2020-06-0109:00didibus:+1: alright, will try it out. Thanks.#2020-06-0109:11didibusThis seems to be the spec in case there is interest: http://wiki.commonjs.org/wiki/Packages/Registry#2020-06-0118:09alexmillerthe idea of using npm to find transitive cljs deps has come up here several times and it's interesting but needs better thinking starting from the goals. tools.deps really has several parts and it seems like the relevant part here is using the tools.deps dependency expander to trace the transitive set of dependencies, starting from a cljs project. tools.deps is designed to be extended for this by implementing various multimethods in the ctda.extensions ns. I've looked at the npm repo and it has everything you would need to do so and would be pretty simple to plug in in that way.#2020-06-0118:10alexmillerhowever, it is then at odds with the rest of tools.deps (and clj) in that the other parts are about making a jvm classpath and I don't think that's what you need or want for cljs.#2020-06-0118:12alexmillerso yes, you could use tools.deps to get a transitive dependency expansion from a heterogenous set of sources including maven, local, git, and npm. I think you want to do that from some cljs tool that then knew what to do with all that stuff to appropriately run the cljs compiler (or whatever).#2020-06-0118:14alexmillerif that big picture made sense, I'd be happy to lay out what needs to be done to plug in an npm procurer into the tda extensions. That part is not hard, but it's not useful unless this in service of some greater vision.#2020-06-0119:04dominicmI think npm is useful outside of cljs. It would make a good alternative to webjars.#2020-06-0119:24alexmilleruseful to do what#2020-06-0119:50dominicmI use webjars to serve assets (js, css) for libraries like codemirror#2020-06-0121:59didibusWoa, webjars already mirrors all of npm and Bower into Jars?#2020-06-0121:59didibusThat means I'm theory, we can already use tools.deps.to.depend on those.#2020-06-0122:00didibusThe only missing piece is from the cljs compiler. But you could easily write a tools.deps alias that extracted the content from the jar and put them in a node_modulrs folder and then created the deps.cljs file. So you could run that, and then run the compilation#2020-06-0122:00didibusHave you found webjars is up-to-date with npm?#2020-06-0122:00didibusAnd mirrors absolutely everything?#2020-06-0122:18dominicmIt's an automated process to request more#2020-06-0120:05hiredmanthose are different things though, one is providing dependencies to the cljs compiler, the other is providing runtime resources#2020-06-0120:11hiredmanit seems like tools.deps is mostly about runtime resources (building a classpath), it just happens that those things almost always overflap significantly in clojure, and people have built tools on top of tools.deps that reinterpret it as build time dependencies (building uberjars out of the stuff it puts on the classpath) because of that overlap#2020-06-0120:12hiredmanthere is, from my very limited perspective on cljs, a lot less overlap between compile time and runtime dependencies there because of the split between the compiler and runtime, and the way js is usually delivered#2020-06-0120:26hiredmanthinking about it, it also strikes me as very unlikely that packages right from npm are suitable to be served up ala webjars, my guess is you need some kind of intervening build step, so really both cases are inputs to some build/packaging process#2020-06-0121:51didibusTo me, this is the same. We want to provide the ClojureScript compiler (which is a Clojure application) the necessary resources that it needs. Because it relied on npm currently, it makes an assumption to not look for them on the classpath, but instead look for them in a node_modules folder. But I feel if it could leverage tools.deps, it's not unrealistic for it to look for them on the classpath instead. With target :bundle it could create a node_modules folder of symlinks to the deps pulled down by tools.deps and thus allow JS bundlers to "find" the tools.deps managed JS dependencies.
Or maybe, tools.deps could just pull the npm deps into that folder directly. Not sure if it's a core part of tools.deps to put all the deps in some central place? Or if it would be fine for it to put them in different places?#2020-06-0122:26alexmillerIt‘a not - it’s up to the procurer to do that, and then tell tools.deps where it is#2020-06-0200:22didibusThat makes it even better then. Anyways, I'm just plenting the seed of an idea. See where things go.#2020-06-0121:54didibusAlso, I'm not super familiar with JS bundlers, do they expect to bundle node_modules stuff? Or could the compiler just copy over all the necessary JS on the output folder and the JS bundler can just bundle it up.#2020-06-0121:54hiredmanthey are not dependencies of the clojurescript compiler, they are dependencies of the code it is compiling#2020-06-0121:54didibusRight, I see them as resources needed by the ClojureScript compiler. I was using dependencies loosely here.#2020-06-0121:59hiredmansticking things on the class path because we have a tool for building class paths does not seem like a considered approach#2020-06-0122:04hiredmanit sounds like the problem you want to solve is transitive js dependencies#2020-06-0122:05didibusWell, personally, I find an explicit list of paths to resources is more considered then an implicit one. But NPM already made that choice. Though here it seems we could make a different one. The only difference is where to look for the resources, do you just scan the classpath, or do you also look for a specific relative folder.
That said, it seems @alexmiller was saying that it would be possible that tools.deps doesn't do classpath generation for those, but just feeds the resolution to something else. So that could be an option as well. So maybe that something else can generate the node_modules folder instead.
That said, I think going with the classpath is best. Because cljs already uses the classpath for ClojureScript deps, and for cljsjs deps. So reconciling those two seem like a nice outcome.#2020-06-0122:10didibusI'm trying to solve JS dependency resolution. So pulling down all required direct and transitive JS dependencies, detecting conflicts between them and letting the user resolve those. That's the first part. Feel tools.deps could do this. Right now the ClojureScript compiler itself is slowly building this functionality inside itself (though it delegates to npm mostly).
The second issue is setting up the environment for running the ClojureScript compiler where it can find your dependencies. It currently relies on a mixture of the classpath + looking for an implicit node_modules folder. I'm not as adamant here, but I still feel if we moved the nodes deps to the classpath as well, it seems to simplify things. Also, it gives us a single place for ordering in terms of which lib to grab first if the JS class exists in more than one place.#2020-06-0122:11didibusBut probably we should discuss these two things separately.#2020-06-0318:48vlaaadnot sure it's important, but I was reminded of a small caching problem I had with tools-deps: it seems to cache the classpath per list of aliases enabled or something similar that excludes the java version, while maven allows selecting different dependencies based on java version right inside the pom#2020-06-0318:49vlaaadso the same deps.edn might produce different classpaths based on JDK used#2020-06-0318:51ghadia repro would be helpful. By design tools.deps doesn't attempt to mimic Maven#2020-06-0318:51vlaaadhttps://github.com/cljfx/cljfx/ is a repro#2020-06-0318:52ghadithat is not a repro, that's a repo#2020-06-0318:52vlaaadgood point#2020-06-0318:52ghadi"`clojure -A:foo -Spath` -> expected this, but received that" is a repro#2020-06-0318:52vlaaadah, sorry, I'll make a repro#2020-06-0319:02vlaaadrepro#2020-06-0319:07alexmiller"switch java version to 8 so it should trigger different profile according to activation rules" is not something clj does#2020-06-0319:07alexmillerthat's a maven thing#2020-06-0319:09alexmilleryou can make Clojure aliases that include the desired classifier variant of cljfx if you like#2020-06-0319:11vlaaadbut it's what it does?#2020-06-0319:11alexmillerit's what Maven does#2020-06-0319:11alexmillerclj is not Maven and I have no plans to add that#2020-06-0319:12vlaaadsure, I understand that is abusing maven features#2020-06-0319:13alexmillerit's very much using Maven features as they were intended#2020-06-0319:13alexmillerjust don't expect clj to support all of that (this is just the tip of the iceberg)#2020-06-0319:17vlaaadOkay, thanks#2020-06-0322:08drewverleeI have deps that specifies a private github repo via SSH (e.g /cdn-cgi/l/email-protection) This works finally locally. But i'm trying to know run my build scripts as part of a deployment pipline (specifically google steps). I'm running into some complications concerning deployment keys, ssh, etc... by any chance has anyone been through this before and have any advice?#2020-06-0323:44hiredmanIt may be easier to use the git build step to fetcch the repo and then treat it as a local dep#2020-06-0323:46hiredmanOr the docs for private git repos might help directly https://cloud.google.com/cloud-build/docs/access-private-github-repos#2020-06-0400:52drewverleeThe downside is that you have to have a GitHub deploy key per repo.#2020-06-0323:52kennyThere's no actual fix mentioned in https://ask.clojure.org/index.php/8725/deps-support-newer-private-file-formats-types-such-ed25519. Anyone know what the fix is? We're onboarding new developers and they all hit this issue. Guessing the fix has something to do with generating an older ssh key?#2020-06-0323:56kennyPerhaps https://ask.clojure.org/index.php/7818/authentication-issues-through-jsch?show=7879#a7879?#2020-06-0400:09kennyGit deps won't work at all unless they generate an older key. I think anyone with a new Mac will hit this issue 😞#2020-06-0400:39ghadied25519 keys work fine if you use an ssh-agent#2020-06-0400:39ghadiIf you rely on .ssh/config you will have a bad time#2020-06-0414:09drewverleecan you point to something that explains why using a config will lead to a bad time?#2020-06-0414:09drewverleeThis seems relevent to some of the waters i'm having to navigate.#2020-06-0414:09drewverleeAs i removed the config and things "just worked"#2020-06-0414:49drewverleeHow can i get more information about why the connector is available?
Cloning:
In the same context i can run git clone on a repo and it succeeds.#2020-06-0414:53drewverleemaybe the docker image i'm running in doesn't have something git deps expects to be installed at a system level?#2020-06-0414:54alexmillerare you running ssh-agent?#2020-06-0414:55drewverleechecking...#2020-06-0414:59ghadiecho $SSH_AUTH_SOCK will tell you definitively#2020-06-0414:59ghadiif you're running in a docker container, probably not, unless you started it explicitly#2020-06-0415:01drewverleethanks ghadi.#2020-06-0415:15drewverleeit doesn't appear to output anything. argh, ill have to somehow start it and save it between google cloud steps.#2020-06-0415:41drewverleeyea. i just jump started the ssh agent first and then called clojure#2020-06-1000:32porkostomusI can use the CLI to launch the Liquid Clojure editor from Windows Powershell, but only if I escape the double-quotes:
PS C:\Users\admin> clj -Sdeps '{:deps {mogenslund/liquid {:mvn/version \"RELEASE\"}}}' -m liq.core
It fails however if I try to use the alias:
PS C:\Users\admin> clj -A:liquid
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate dk/salza/liq/core__init.class, dk/salza/liq/core.clj or dk/salza/liq/core.cljc on classpath.
Full report at:
C:\Users\admin\AppData\Local\Temp\clojure-4978726344790284382.edn
My deps.edn entry:
:liquid
{:extra-deps {mogenslund/liquid {:mvn/version "RELEASE"}}
:main-opts ["-m" "dk.salza.liq.core"]}
All the other aliases seem to work fine.#2020-06-1000:41seancorfield@porkostomus Looks like the project changed it's main namespace (from dk.salza.liq.core to just liq.core)#2020-06-1000:42seancorfieldI'll update my dot clojure file#2020-06-1000:45seancorfield@porkostomus I fixed the alias in my repo -- if you cloned that, you can just git pull to get the latest#2020-06-1000:56porkostomus@seancorfield Great, thanks!#2020-06-1021:34Aaron CummingsI have just upgraded from 1.10.0.442 to 1.10.1.536, and I've found a change in behavior. In .442 I had changed my $INSTALL/lib/clojure/deps.edn to point the central and clojars repos to a local partial mirror. In .536, the same change has no apparent effect. Here is an example of my lib/clojure/deps.edn:
{
:paths ["src"]
:deps {
org.clojure/clojure {:mvn/version "1.10.1"}
}
:aliases {
:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.8.677"}}}
:test {:extra-paths ["test"]}
}
:mvn/repos {
;; "central" {:url ""}
;; "clojars" {:url ""}
"central" {:url "file:///home/aaron/cljtest/central/"}
"clojars" {:url "file:///home/aaron/cljtest/clojars/"}
}
}
The reason I'm doing this is our development machines at work do not have internet access, so I determine in advance all of our central and clojars dependencies and cache those locally in NFS.
Is there a different way that I should override central and clojars default locations?#2020-06-1021:37alexmillerthat file is no longer used - it's embedded in a resource in the uber jar#2020-06-1021:37alexmilleryou can set those in your ~/.clojure/deps.edn though and should have the same effect#2020-06-1021:42Aaron CummingsOkay, I'll give that a try.#2020-06-1021:43alexmiller(that would have worked on older version too)#2020-06-1021:44Derek Passenon that note, when I run clj -Sverbose on 1.10.1.536, the config_paths still lists the system install deps.edn file#2020-06-1021:44Derek Passenon that note, when I run clj -Sverbose on 1.10.1.536, the config_paths still lists the system install deps.edn file#2020-06-1022:01alexmilleryeah, it's hard-coded in there and hasn't been updated#2020-06-1022:02alexmillerthere are actually some tools that rely on this output and do (now incorrectly) still use the file so I have been delaying on removing it#2020-06-1022:02Derek Passenmakes sense#2020-06-1021:51Aaron CummingsSuper, that did work. You're right @dpassen1 - the -Sverbose flag output did lead me astray.
I did like the file in the installation - that let me setup for all of my users in one place. But I'll switch us over to the ~/.clojure/deps.edn instead.#2020-06-1021:57seancorfieldGiven that the system deps is meant to be stable and immutable, I think it's good for it to no longer be an actual file.#2020-06-1021:58Derek PassenAgreed Sean#2020-06-1021:58Derek PassenThere’s a reason for ~/.clojure/deps.edn and CLJ_CONFIG env var#2020-06-1021:58Derek Passenat least i think it’s CLJ_CONFIG#2020-06-1021:59seancorfieldIt is. We use that at work to point to a repo-wide deps file, and then each subproject in the repo has its own deps file.#2020-06-1022:00Derek PassenI probably learned it from your mention somewhere 🙂#2020-06-1022:00seancorfieldCLJ_CONFIG=../versions clojure -A:defaults ... -- the :defaults alias pulls in all the :override-deps from versions/deps.edn#2020-06-1023:31Aaron CummingsI had forgotten about CLJ_CONFIG - that seems like the better solution to what I'm doing. Thanks for the pointers everybody.#2020-06-1119:27nhaI am looking for something like a maven api that I can call from a Clojure app to make a .jar from a java file + it’s main class name.
I’ve never used maven directly (at least not in anger). Any opinions here?
For context: I am making a .jar and uploading it to a lambda-like service.
Not about tool deps so I hope this is OK - seems like the place where I am most likely to get answers 🙂
Please reply in the thread#2020-06-1119:27nhaI am looking for something like a maven api that I can call from a Clojure app to make a .jar from a java file + it’s main class name.
I’ve never used maven directly (at least not in anger). Any opinions here?
For context: I am making a .jar and uploading it to a lambda-like service.
Not about tool deps so I hope this is OK - seems like the place where I am most likely to get answers 🙂
Please reply in the thread#2020-06-1119:27nhaPlease reply here to keep the channel tidy 🙂#2020-06-1119:28dominicm@U0ALP2929 What service are you using? I created an AWS lambda-specific output for pack and would be interested in targetting others.#2020-06-1119:29nhait’s actually pulsar Functions https://pulsar.apache.org/docs/en/functions-overview/#2020-06-1119:30nhaSo hence lambda-like, it’s not exactly the same. However they accept a
--jar target/my-jar-with-dependencies.jar \
--classname org.example.functions.WordCountFunction \
#2020-06-1119:31dominicmOK, looks like you need an uberjar of some kind in that case. I would recommend the library @U04V70XH6 maintains for that.#2020-06-1119:31dominicmThere's downsides to uberjars (which is why pack doesn't produce them), I'm curious to know if they accept anything else, so I'm reading the docs now :)#2020-06-1119:34nhais this the lib? https://github.com/seancorfield/depstar#2020-06-1119:37dominicmThat's the one :)#2020-06-1119:37dominicmDoesn't look positive for alternative forms. Unfortunate. Depstar is what I'd recommend :)#2020-06-1119:41nhadepstar looks like it is supposed to be used from the CLI, and not sure it supports .java files?
I am looking for something to embed. I could always shell it / reach into the namespaces though#2020-06-1119:43nhawell the api looks simple enough simple_smile https://github.com/seancorfield/depstar/blob/master/src/hf/depstar/jar.clj#L9#2020-06-1119:43dominicmOh, sorry, I thought you wanted to make a jar for clojure#2020-06-1119:43dominicmapologies#2020-06-1119:43dominicmRight, you don't need much at all#2020-06-1119:43dominicmjavac MyFile.java then zip jar.jar MyFile.java#2020-06-1119:44dominicmthat's it, you're done!#2020-06-1119:44dominicmthere might be an official java tool for making a jar, but zip works OK#2020-06-1119:44seancorfield@U0ALP2929 To build uberjars with depstar, you should be able to depend on the library and call hf.depstar.uberjar/run with the appropriate options -- see how the -main function sets those up. It assumes there's a pom.xml file available because it uses the presence of that to trigger some of the AOT and manifest generation stuff. It assumes, at a minimum, you've run clojure -Spom in a project, but you could probably generate a stub pom.xml as a temp file and pass that in.#2020-06-1119:45seancorfieldAnd depstar works from the classpath -- so it will build a JAR from whatever is on your classpath which is very likely not what you're looking for 🙂#2020-06-1119:45seancorfieldjar is the "official Java tool for making a jar" but it's basically just zip 🙂#2020-06-1119:46nhaAhah I see 🙂 not what I am looking for indeed. Shelling to javac or even mvn looks like it could be the way then#2020-06-1119:46nhaI found this https://maven.apache.org/ref/3.1.0/maven-embedder/index.html but I fear it may be a can of worms 😄#2020-06-1216:57TravisI am currently attempting to use the new Google Artifact Maven Registry in Google cloud which has a custom maven wagon. Is this possible to use with tools-deps ?#2020-06-1217:02alexmillerif it needs a custom wagon, then no#2020-06-1217:02alexmillerwhy does it need a wagon?#2020-06-1217:02alexmillergot a link?#2020-06-1217:11TravisI believe its to handle the security mostly to connect to it. Uses the underlying gcloud ( cli ) creds#2020-06-1217:12Travishttps://cloud.google.com/artifact-registry/docs/java/quickstart#2020-06-1217:13alexmilleryeah, that was my guess, might be be some storage access optimizations over just http too#2020-06-1217:13alexmillerwell, feel free to log this as a request at https://ask.clojure.org so people can vote on it#2020-06-1217:13Travisok, thanks#2020-06-1217:13alexmillerif it's a wide enough need, will consider#2020-06-1305:54sveriHi, Is there a way to make sure I download needed dependencies for a givem library? Like: clojure -Sdeps '{:deps {seancorfield/depstar {:mvn/version "1.0.94"}}}' but without opening a repl?#2020-06-1306:06seancorfield@sveri Add -e nil to the end of that command.#2020-06-1306:07seancorfieldIt will perform all the downloads etc and then evaluate nil (and not print it because it only prints non-`nil` values)#2020-06-1306:08seancorfield-Stree and -Spath will also circumvent a REPL and will show you the dependency tree or class path respectively, if you want some confirmation of the resources being accessed.#2020-06-1306:09sveriThank you, exactly what I am looking for. I don't need the extra output for now.#2020-06-1306:09sveri@seancorfield#2020-06-1308:20sveri@seancorfield I hope it's fine if I ask you directly here. I have been using depstar successfully so far and am experimenting with graalvm right now. It seems like native-image analyzes faster if direct linking is enabled. Is there a way to pass the :jvm-opts ["-Dclojure.compiler.direct-linking=true"] for aliases, if I read that right.#2020-06-1318:30seancorfield@sveri Or you can pass it on the command-line as -J-Dclojure.compiler.direct-linking=true#2020-06-1319:05sveri@seancorfield Does that -J-D... work for clj calls too? In only saw it in reference to the native-image tool#2020-06-1319:06seancorfieldYes, -J is a clojure/`clj` option to pass JVM options.#2020-06-1320:22sveriOkay, great, thank you.#2020-06-1522:44alidlorenzotools.deps doesn’t load :local/repo aliases, right? how do you handle conditionally loading dev/test/etc aliases on per subproject basis?#2020-06-1522:44alexmillerRight, and you don’t #2020-06-1522:45alexmillerSomething we’ve thought about a little #2020-06-1522:45alexmillerBut needs more thinking#2020-06-1522:46alidlorenzoyeah, I found some archived slack convos about it. kind of hard to appropriately work with monorepos without such a feature, everything has to configured in the root deps.edn (?)#2020-06-1523:03alidlorenzowhat if it aliases were specifically included, just like dependencies are?
{my-subproject {:local/root "./my-subproject"
:aliases [:dev :test]}}#2020-06-1523:19alexmillerI’m not going to start from what if - this needs a design process#2020-06-1523:25seancorfieldWe have a monorepo at work and we specifically use CLJ_CONFIG to set the "user" deps.edn to come from a specific directory in the monorepo so that we have a single, central place for all repo-wide aliases anyone needs. This also helps maintain repeatability because each developer's "user" deps.edn is excluded and we can control exactly what each alias does.#2020-06-1523:31alidlorenzoif subprojects have paths that are loaded based on alias, do you configure this all in the central deps.edn instead?
e.g {:extra-paths ["subproject1/test" "subproject2/test"]}#2020-06-1523:52seancorfieldIt depends on whether the alias (and the optional subproject) are going to be needed for "all" other subprojects or just specific one(s). We scope things appropriately.#2020-06-1523:53seancorfieldAll our subprojects are at the same level so ../subproject/path always works.#2020-06-1609:18robert-stuttafordapologies if an FAQ (i did try to google this) - should :git/url work if the target sha is in a feature branch? so tools.gitlibs appears to be failing silently - situation: i see the gitlibs' "Checking out: " message, but no errors, and then when tools.deps tries to use the checkout, it fails because it's not there. i checked ~/.gitlibs manually and found the folder to be missing.#2020-06-1609:58borkdude@robert-stuttaford yes. have you tried -Sforce? I find myself needing that occasionally with git deps#2020-06-1611:49robert-stuttaford-Sforce doesn't seem to make a difference, unfortunately, thanks @borkdude#2020-06-1611:51alexmillerMight be something wrong with object dir - you could try rm’ing that under gitlibs#2020-06-1611:53alexmillerCan’t say I’ve seen this kind of problem before#2020-06-1611:55vlaaadmaybe feature branch isn’t pushed? 🙂#2020-06-1611:56alexmillershould fail if sha isn’t known#2020-06-1612:19robert-stuttafordi have one other hash in the folder, so it's defo succeeded at least once before. i'll scrub and retest! (feature branch defo pushed haha)#2020-06-1612:22robert-stuttafordthe commit hash that succeeded is on the default github branch (`develop` in our case), whereas the one that failed is in a feature branch. could it be that the git checkout it's doing isn't able to find the feature branch's commit?#2020-06-1612:24alexmillerShouldn’t matter at all, I do this all the time#2020-06-1612:24robert-stuttafordscrubbing and retesting yields the same results#2020-06-1612:25alexmillerYou sure you have a commit sha and not like a tree sha or something#2020-06-1612:26robert-stuttafordyes - i can visit the sha on https://github.com/org/repo/commit/SHA and i have it checked out locally#2020-06-1612:26robert-stuttafordChecking out:
the first line is one that works (it's there after i run), the second line is the one that's meant to work but doesn't (not there after i run), and the third line is the downstream failure due to the missing folder.#2020-06-1612:27robert-stuttafordi guess the next step is to dig with tools.gitlibs at the repl!#2020-06-1612:27alexmillerYeah that definitely drops down a layer#2020-06-1612:28robert-stuttafordglad to know that this should work, thank you#2020-06-1612:28alexmillerLet me know how you make out#2020-06-1612:28robert-stuttafordinterestingly this resolves fine on our CI instance, but not locally 🤔 on the CI, i see Skipping coordinate: with the same sha, presumably because it's already got it. of course, good old :local/root works locally.#2020-06-1715:24mokrHi, I’m not sure where to start explaining to get the right context, but for short I’m trying to convert a somewhat large, working leiningen project to tools.deps. I’ve got the tools.deps + shadow-cljs dev experience working, but I’m struggling with the uberjar part.
The project is based on a Luminus template from 2017 so I haven’t done this part myself and I’m not an experienced Java developer either.
Right now I’m using this tool and guide (https://github.com/tonsky/uberdeps), but clj -Aclj -C:assets -e "(compile 'mtools.core)" (“clj” and “assets” are actual aliases) seems to just hang and nothing is created in the “classes” dir. At the terminal I see some output from a (println) in a middleware ns and some WARNING: An illegal reflective access ... and similar.
Any tips to help me along the way? E.g. how to get more verbose output from that compilation stage?#2020-06-1715:40hiredmanHanging when running compile usually indicates you have a top level form doing something that blocks/takes a long time#2020-06-1715:42mokrI’m quite confused here. Does it run the code when it compiles? I’m surprised to see that println output as well to be honest.#2020-06-1715:43hiredmanIt does#2020-06-1715:43hiredmanAot compilation is the same as regular code loading, it just saves the bytecode to disk as it generates it#2020-06-1715:45hiredmanLikely your lein project was never aot compiled, because you would have experienced the same thing there#2020-06-1715:46mokrI’m quite confident that it was. Lein profile has :aot :all and after extracting the jar I see lots of .class files#2020-06-1715:47mokrSo when it hangs it might be the Immutant web server doing its job, waiting for connections?#2020-06-1715:48hiredmanThen you've made other changes than just switching from lein to deps.edn#2020-06-1715:48hiredmanIt could be, it depends how you are starting it#2020-06-1715:49hiredmanI believe luminus uses mount to manage how and when things are started#2020-06-1715:50mokrI have certainly made quite a few changes, but not “that much” to that part of the code.#2020-06-1715:50mokrYes, it uses mount.#2020-06-1715:50hiredmanPast tense?#2020-06-1715:50mokrtypo#2020-06-1715:52hiredmanIf you have something like (startmount) as a top level form, it will start mount while running compile#2020-06-1715:52mokrDue to shadow-cljs liking to have all it’s source paths in one listing, I’m having to move away from the “env” approach used in Luminus. That is: Luminus uses source path to include different versions of a namespace in dev vs prod.#2020-06-1715:52mokrI might have messed something up in that ongoing process.#2020-06-1715:52hiredmanIf you have a user.clj that starts mount when loaded#2020-06-1716:00mokrNo user.clj, but mount is in core.clj that I call as main.
Mount has actually bitten me quite a few times.#2020-06-1716:02mokrWould have been way easier if my project was open source.
So, one way to get further is to start adding some printlns to my code, maybe?#2020-06-1716:05mokrThanks, @hiredman at least I did learn something here. It’s time for dinner so I’ll be away for a while.#2020-06-1814:33mokrJust for information in regards to my question above:
I got the uberjar generation with AOT working after switching to https://github.com/seancorfield/depstar and doing AOT by passing the -C option.
It was a lot of trial and error so maybe I could have gotten https://github.com/tonsky/uberdeps to work as well.
On top of of this I surfaced an issue that has been in my code for a long time: I run my prod server on port 6000, but remap it in docker-compose. Due to that I have never experienced that browsers (Chrome, FF, …) https://superuser.com/questions/188058/which-ports-are-considered-unsafe-by-chrome, before I now attempted to run the jar locally and couldn’t connect.#2020-06-1819:06markbastianIs there deps.edn analogue of the lein-environ plugin? I.e. a facility for setting an env map in a sidecar file (or your global deps.edn file) that is merged with the global env?#2020-06-1906:37rickmoynihanYou could try using: https://direnv.net/#2020-06-1819:12alexmillernot to my knowledge#2020-06-1819:13markbastianIs it possible to set env vars in deps.edn or is that not really what it's meant for?#2020-06-1819:13alexmillerit's not possible right now#2020-06-1819:13alexmilleryou can of course just set them on the call to clj using normal shell methods#2020-06-1819:14alexmillerI wouldn't rule out the possibility of adding something like that in the future#2020-06-1819:14markbastianYeah, just trying to figure out the "right" way to set them when launching a REPL from Cursive. Using the Cursive config works fine for now.#2020-06-1819:15alexmillerin clj, one option is to create an alias that adds an :extra-paths and then load config via a resource#2020-06-1819:15alexmilleran alias can also set Java system properties#2020-06-1819:16alexmillerso those are two paths to have an alias add extra config#2020-06-1819:17markbastianThat makes sense. "an alias can also set Java system properties" sounds promising. Thanks!#2020-06-1819:17alexmillerbut be careful that you're not putting sensitive values in your deps.edn#2020-06-1819:18alexmilleryou'd want to have a system property that point to a file (or env var) to read a sensitive value#2020-06-2315:31pmonksG’day! Does anyone happen to know of an equivalent of lein-licenses (https://github.com/technomancy/lein-licenses) for tools.deps? Either my google-fu is weak, or such a thing doesn’t exist yet.#2020-06-2317:14seancorfieldI'd never heard of such a thing, even for Leiningen before. Looking at the code, I don't think it would be too hard to write a deps.edn-based tool that did the same thing if someone felt sufficient need...#2020-06-2320:23pmonksYeah that was my thought too, and it helps a lot with license compliance (which has been a big deal most of the places I’ve worked).#2020-06-2320:25pmonksIs there a standard place in a deps.edn to put license information? Maven offers such a thing, though it’s not mandatory so not universally adopted (which is a shame).#2020-06-2320:37seancorfieldThere is nowhere in deps.edn to put such a thing. The repo might have pom.xml and you'd look in there, or look for a LICENSE file of some sort in the project root perhaps.#2020-06-2320:39seancorfieldFor example: https://clojars.org/seancorfield/next.jdbc -- display the license type because it's in the pom.xml file: https://github.com/seancorfield/next-jdbc/blob/develop/pom.xml#L10-L15#2020-06-2320:41seancorfieldBe aware that clojure -Spom does not add <licenses> into the generated pom.xml -- I have it because I start my projects off with clj-new and the templates for pom.xml in app, lib, and template all have a default pom.xml file with an EPL 1.0 license listed in them.#2020-06-2322:19pmonksRight, and most of the tools that do this kind of thing “for real” rely on source repository analysis to determine licensing, rather than (potentially incorrect) declarations in build files like pom.xml.
The challenge in the Java ecosystem is that for dependencies often all you can reliably obtain is a pom.xml (and associated JAR(s)). That’s all lein-licenses uses, for example.#2020-06-2322:22pmonksThis is (part of) the problem that the https://spdx.dev/ is trying to solve, and I’m a bit of an advocate for their efforts. They would get a huge boost if language ecosystems (such as Clojure’s) were to adopt SPDX for licensing information.#2020-06-2322:24pmonksWhile they’re better than some wild west ecosystems (cough Javascript cough), the JVM-based ecosystems are still a bit too cavalier in their approach to stating and disseminating licensing information, imho…#2020-06-2321:10drewverlee{}
Is a valid deps.edn right?#2020-06-2321:11seancorfieldYup.#2020-06-2321:11seancorfieldExactly the same as not having one.#2020-06-2321:12drewverleeWhat could cause two invocations of a the same clj command to result in different outcomes? i get a error building my claspath "dont know how to create an iseq from symbol" and everyone else gets the desired result.#2020-06-2321:13drewverleesomething in my local maven repo?#2020-06-2321:15drewverleehttps://gist.github.com/drewverlee/6a65af3d3b586b7bc272c3a4c2d90944 command and ST for those interested 🙂#2020-06-2321:16drewverleeclojure 1.10.1#2020-06-2321:18drewverleei blew away my m2 repo and got the same result.#2020-06-2321:21drewverleethat command as a missing curly bracket#2020-06-2321:21drewverleenevermind, its fine#2020-06-2321:24drewverleemy argument order was wrong#2020-06-2321:24drewverleei needed to pass -Srepro first then Sdeps#2020-06-2321:38seancorfield-Sdeps takes an argument, which is the {:deps ...} EDN.#2020-06-2321:39seancorfieldSo it's not so much that you need -Srepro first as that you need the EDN immediately following -Sdeps#2020-06-2323:53colliderwriterI have a repo with multiple related projects in it which i'm moving to tools.deps. Is there a way to include an external edn file so i can keep my aliases and common paths DRY?#2020-06-2400:04seancorfield@colliderwriter What we have done at work is to take advantage of the (undocumented?) CLJ_CONFIG environment variable to point the "user deps" path at a directory in our monorepo and have all the common aliases, overrides, etc in a deps.edn file in there -- including aliases for all the tooling needed for working across all the projects in our monorepo.#2020-06-2400:07seancorfieldSo our repo structure is:
clojure
|- versions (the common deps folder)
|- project-a
|- project-b
|- ...
and then we cd into a project to work and run clojure like this:
$ CLJ_CONFIG=../versions clojure -A:some:aliases and args
and it uses versions/deps.edn instead of a user-level deps.edn file plus the deps.edn from the project-x folder that we are running in.#2020-06-2400:20colliderwriterthanks for that. it will get the gears to mesh so i can move on today, but i foresee a problem with devops once i try to hook that up because bazel/rules_clojure won't know how to do that. This seems like a missing feature in tools.deps, especially because this is possible via lein-parent in old money. I guess i should go look at the I wish there were an issue list to see if anyone else has asked for this.#2020-06-2400:34seancorfield@colliderwriter You mean this https://clojure.atlassian.net/projects/TDEPS/issues ?#2020-06-2400:35colliderwriterI sure do. Thanks again.#2020-06-2400:35seancorfieldGiven that it is driven via an environment variable, and that could be automated via a simple "build" shell script in the repo, I don't see why it would cause problems for devops.#2020-06-2400:36seancorfieldIt has the benefit that you can control all tools directly within the repo since that "main" deps.edn file is under version control with everything else.#2020-06-2400:41colliderwriterI absolutely agree that the shared files are in the right place. I'm already using Bazel to do my builds and it is pretty opinionated. I'll have to look into my options there.#2020-06-2401:43hiredmanI don't know bazel, but I suspect slot of those 'srcs' instead of being lists of source files should be directories containing source files#2020-06-2401:47hiredmanclj, lein, boot, most clojure tooling has you specify what ends up being classpath roots (directories of classes or source code, and jar files via deps) not individual source files#2020-06-2400:43colliderwriterin case anyone's curious https://github.com/simuons/rules_clojure#2020-06-2400:47seancorfieldWow, that looks horribly restrictive. I'm sorry 😐#2020-06-2402:26hiredmanIt is kind of a shame to have to explicitly list intra mono repo dependencies, given a closed mono repo world you can derive a project dependency graph from the namespace dependency graph#2020-06-2509:19danielcomptonDoes tools-deps support specifying version ranges? I couldn’t see anything about it. It looks like it supports it when reading a POM as a transitive dependency#2020-06-2511:33dominicmI'm pretty sure it doesn't#2020-06-2511:33dominicmBy design#2020-06-2511:37dominicmI'm lying https://github.com/clojure/tools.deps.alpha/blob/0ceb075c24ce28d4ec22027aa141cb19baad1b69/src/main/clojure/clojure/tools/deps/alpha/extensions/maven.clj apparently support was added#2020-06-2511:37dominicmIt just gets the highest version in the range#2020-06-2514:05murtaza52I have put my aliases in the ~/.clojure/deps.edn file (I am suing the dot file from seancorfield), but when I try to call clj -A:new app test/one , I get the following error -
Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
app (No such file or directory)
Full report at:
/tmp/clojure-3392454598929432067.edn
It seems it is not able to find the alias, because when I inline the alias and call the command it works. Why is it not recognizing the ~/.clojure/deps.edn file ?#2020-06-2515:34seancorfield@murtaza52 Did you also get this WARNINg about that error message?
WARNING: Specified aliases are undeclared: [:new]
What does clojure -Sdescribe show for you?#2020-06-2515:36murtaza52~ >>> clojure -Sdescribe [1]
{:version "1.10.1.536"
:config-files ["/usr/local/lib/clojure/deps.edn" "/home/murtaza52/.config/clojure/deps.edn" ]
:config-user "/home/murtaza52/.config/clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/murtaza52/.config/clojure"
:cache-dir "/home/murtaza52/.config/clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}#2020-06-2515:37murtaza52@seancorfield No I did not get that warning#2020-06-2515:39murtaza52I moved the file /.clojure/deps.edn to /.config/clojure/deps.edn, but I am still getting the same output as above.#2020-06-2515:46seancorfieldHow did you install the Clojure CLI tools? I've never seen it try to use ~/.config/clojure as the user-level location...#2020-06-2515:47seancorfieldWhat O/S is this on? What is your HOME environment variable? (`echo $HOME`)#2020-06-2515:48murtaza52~ >>> echo $HOME
/home/murtaza52#2020-06-2515:48borkdudethis is caused by xdg-config stuff on linux right#2020-06-2515:48murtaza52This is manjaro linux#2020-06-2515:48borkdude@seancorfield https://github.com/borkdude/deps.clj/blob/5d51b80176bdcb96927ce5e6aedb25581db0e2e0/src/borkdude/deps.clj#L341#2020-06-2515:51murtaza52sorry I m still lost, how should I fix it ? where does that deps.clj go ?#2020-06-2515:51seancorfieldIt's a separate tool. He's just showing that the XDG thing is a problem.#2020-06-2515:51murtaza52this is a new install of manjaro without a DE, I had installed it before with DE and it worked fine.#2020-06-2515:51borkdudeyeah, I should point to the bash script, this is a clojure port of it#2020-06-2515:52borkdudecan you do echo $XDG_CONFIG_HOME?#2020-06-2515:52murtaza52this one just bspwm as the tiling manager#2020-06-2515:52murtaza52~ >>> echo $XDG_CONFIG_HOME
/home/murtaza52/.config#2020-06-2515:52borkdudeand then try with the -Sforce with your changes in place#2020-06-2515:53borkdudeyes, so it will look for /home/.../.config/clojure/deps.edn#2020-06-2515:54murtaza52but in already has a dir called /.config/clojure , should I replace it with /.config/.clojure ?#2020-06-2515:55borkdudeno sorry, without the dot, it was a typo on my behalf#2020-06-2515:55borkdudeand then re-try your action with -Sforce#2020-06-2515:56murtaza52what does sforce do ? yes it worked !#2020-06-2515:56borkdudeit ignores a cached file#2020-06-2515:56murtaza52cool thanks !#2020-06-2515:58seancorfieldNew topic: tools.deps.alpha ignores :scope in a coordinate, yes? That's a Maven thing that Lein/Boot support but is not relevant for deps.edn...?#2020-06-2516:01seancorfieldHahaha... well, someone opened a bug against depstar about it putting :scope "provided" dependencies into the JAR and when I created a quick test to check the classpath clojure -Spath in a new project with a :scope "provided" dependency, yes, it's on the classpath so this is t.d.a behavior.#2020-06-2517:23souenzzohello. I'm trying to use git deps with aws codecommit, inside a ec2 instance but I get
Error building classpath. : Authentication is required but no CredentialsProvider has been registered
but when I (inside the ec2 instance) git clone it works
Anyone else using tools_deps + codecommit?
(stacktrace on comment)#2020-06-2517:23souenzzoCloning:
Error building classpath. : Authentication is required but no CredentialsProvider has been registered
org.eclipse.jgit.api.errors.TransportException: : Authentication is required but no CredentialsProvider has been registered
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:248)
at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:306)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:200)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:50)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:110)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
at clojure.tools.deps.alpha.extensions.git$eval999$fn__1001.invoke(git.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:169)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:237)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
Caused by: org.eclipse.jgit.errors.TransportException: : Authentication is required but no CredentialsProvider has been registered
at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:530)
at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:345)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:137)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:123)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1269)
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:237)
... 37 more#2020-06-2517:32hiredmanjgit may not support whatever authentication method git is using there#2020-06-2517:44ghadiyou need to use ssh auth, not https#2020-06-2518:17souenzzohttps://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_ssh-keys.html
To use ssh, i need to create a key and autorize it. That's the path @U050ECB92?#2020-06-2518:17ghadiyes#2020-06-2518:17ghadiand put the key in an ssh-agent#2020-06-2520:19souenzzo@U050ECB92 that't not aplicable once I'm trying to use LaunchConfig#2020-06-2519:26murtaza52If I am using a template to generate a project, and the template has changed, will just using -Sforce force the new template to be retrieved ?#2020-06-2519:28seancorfieldNo. The template is only used when the project is first created.#2020-06-2519:29seancorfieldAfter that, it's just "your project" and no longer related to the template.#2020-06-2519:53murtaza52nope want to create a new project with the template, and the template was recently updated, how do I ensure that the updated template from clojars will be pulled for the new project ?#2020-06-2519:54seancorfieldBy default, clj-new always uses the most recent stable version of a template.#2020-06-2519:55seancorfieldPer the clj-new documentation (README), you can override that behavior:
-S or --snapshot -- look for -SNAPSHOT version of the template (not just a release version)
-v or --verbose -- enable debugging -- be verbose! -vv and -vvv are increasingly verbose
-V or --version, followed by a version -- use this specific version of the template
#2020-06-2519:58murtaza52cool thanks#2020-06-2522:00dvingo@seancorfield I'm trying to debug the template @murtaza52 is using - is there a way to determine what clj-new will interpret as the "most recent version" I read through this: https://docs.oracle.com/middleware/1212/core/MAVEN/maven_version.htm#MAVEN8855
and this answer (https://stackoverflow.com/questions/13004443/how-does-maven-sort-version-numbers) led me to org.apache.maven.artifact.versioning.ComparableVersion
I tried deploying new versions that I confirmed ComparableVersion should sort as "new" but I'm not seeing them when I try using the template#2020-06-2522:05seancorfield@danvingo It will use whatever version t.d.a. would select if it were a dependency with a "RELEASE" version.#2020-06-2522:05seancorfieldWhat is the template you're trying to work with?#2020-06-2522:05dvingohttps://clojars.org/dv.fulcro-template/clj-template#2020-06-2522:06dvingoI wanted to use dates as the versions... I'm just learning about the wonderful world of maven version semantics#2020-06-2522:07seancorfieldI posted the options above that would let you do that.#2020-06-2522:08seancorfieldWhen I tried it -- having never looked at that template before -- here's what I got:
(! 709)-> clj -A:new dv.fulcro-template dvingo/example -vvv
Downloading: dv/fulcro-template/clj-template/maven-metadata.xml from clojars
Output from locating template:
Downloading: dv/fulcro-template/clj-template/2020-06-25-4/clj-template-2020-06-25-4.pom from clojars
Downloading: dv/fulcro-template/clj-template/2020-06-25-4/clj-template-2020-06-25-4.jar from clojars
Generating fresh 'clj new' dv.fulcro-template project.
#2020-06-2522:10dvingoah thank you! sorry I overlooked that#2020-06-2522:10seancorfieldAnd here's how to pull down specific versions:
(! 714)-> clj -A:new dv.fulcro-template dvingo/example -V 2020-06-25-4
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:09:24
(sean)-(jobs:0)-(~/clojure)
(! 715)-> clj -A:new dv.fulcro-template dvingo/example2 -V 2020.06.3
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:09:55
(sean)-(jobs:0)-(~/clojure)
(! 716)-> clj -A:new dv.fulcro-template dvingo/example3 -V 2020-06-25-3
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:10:29
(sean)-(jobs:0)-(~/clojure)
(! 717)-> clj -A:new dv.fulcro-template dvingo/example1 -V 1.0.0
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:10:41
(sean)-(jobs:0)-(~/clojure)
(! 718)->
#2020-06-2522:13dvingohmm, I'm on a machine that has prior versions of the template in .m2 and I'm not seeing what you're seeing:
$ clj -A:new dv.fulcro-template dvingo/example -vvv
Output from locating template:
Generating fresh 'clj new' dv.fulcro-template project.
#2020-06-2522:14seancorfieldWell, you won't see any output if it doesn't have to download anything.#2020-06-2522:15dvingoIt at some point stopped downloading the newest versions:
Thu Jun 25, 2020 18:05:15:~/.m2/repository/dv/fulcro-template/clj-template ()
$ l
total 24
drwxr-xr-x 9 default staff 288B Jun 25 18:12 ./
drwxr-xr-x 5 default staff 160B Jun 3 13:01 ../
drwxr-xr-x 7 default staff 224B Jun 23 21:47 2020-06-23-2/
drwxr-xr-x 7 default staff 224B Jun 25 13:31 2020-06-24/
drwxr-xr-x 7 default staff 224B Jun 25 13:56 2020-06-25/
drwxr-xr-x 7 default staff 224B Jun 25 14:05 2020-06-25-2/
-rw-r--r-- 1 default staff 918B Jun 25 14:05 maven-metadata-clojars.xml
-rw-r--r-- 1 default staff 40B Jun 25 14:05 maven-metadata-clojars.xml.sha1
-rw-r--r-- 1 default staff 280B Jun 25 18:12 resolver-status.properties
#2020-06-2522:16seancorfieldI think t.d.a. only checks once per day for releases.#2020-06-2522:16dvingoahhh, ok#2020-06-2522:17seancorfieldIt's very unusual to have multiple releases on the same day. Even with snapshots.#2020-06-2522:18seancorfieldIf you blow away ~/.m2/repository/dv/fulcro-template/clj-template then it should download whatever is the latest version.#2020-06-2522:18dvingoyea that makes sense to save time for the usual case. I wasn't aware of how the interaction with maven occurred - thanks for the help!
deleting the dir from the .m2 repo made it pull the latest#2020-06-2522:18seancorfieldYup, that 🙂#2020-06-2522:19seancorfieldLike I said above, you can always use the -V / --version option to download a specific version.#2020-06-2522:19dvingoright -thanks again!#2020-06-2522:22seancorfieldYou may be able to use a version range like this -V '[0,)' but, again, Aether (the Maven-stuff underneath t.d.a.) may only check once a day.#2020-06-2522:24seancorfield(FWIW, I downloaded 2020-06-25-2 per your directory listing above, then used [0,) and it did pull down 2020-06-25-4... @danvingo#2020-06-2522:25seancorfieldSo it may be that the range checks every time? (maybe a Maven expert can confirm/deny that?)#2020-06-2522:26dvingogotcha - my main use case is template users and not wanting to have them specify versions
i'll just use ComparableVersion at the repl before sending a new version 🙂#2020-06-2522:26seancorfield(hard to tell -- having pulled down the -2 version, running it without a version still pulled down -4 for me, based on RELEASE instead)#2020-06-2522:27seancorfieldIf you're not doing more than one release a day, I suspect you won't have a problem 🙂#2020-06-2522:28dvingoperfect, thanks!#2020-06-2522:36seancorfieldOne final thing @danvingo if you use the -S option, it uses a version range of (0.0.0,)#2020-06-2522:38dvingoahh good to know.
I think the convention I wanted to use is working - 2020-06-25 compares -1 to 2020-06-25-1 and 2020-06-25-1 comes before 2020-06-25-2
so looks like the issue was maven not being checked by t.d.a. and not version sorting#2020-06-2522:40seancorfieldThis is partly why I've moved to MAJOR.MINOR.COMMITS so I don't have to worry about that. COMMITS is always increasing.#2020-06-2522:43dvingomakes sense - I just want to do my part to help kill semantic(less) versioning (I know you're not following that, per the note in the readme, but it still follows the syntax)#2020-06-2522:48seancorfieldAgreed. There are plenty of ways to do it. YYYY-MM-DD.SEQ is as good as any -- and there are plenty of libraries out there, even in the Java world, that use date/time versions instead of semantic(less) versioning (I like that term... Imma gonna stealz it!)#2020-06-2601:08dvingoawesome, that's good to hear, and yes pls do!#2020-06-2601:24dpsuttonand if you are wondering, the comparison seems to be https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions/maven.clj#L88-L97#2020-06-2617:06pmonksSlightly OT, but does anyone have any good links to further reading on why semantic versioning is problematic? (and yes, I’ve watched the video of Mr Hickey’s presentation on the topic)#2020-06-2617:13hiredmanhttps://gist.github.com/jashkenas/cbd2b088e20279ae2c8e#2020-06-2617:38seancorfieldThe linked issue is definitely worth a read too https://github.com/jashkenas/underscore/issues/1805 (it's so long that GH "helpfully" hid 50+ comments that I had to click to expand!).#2020-06-2617:51pmonksThanks!#2020-06-2617:58pmonksSo criticism is great, but what’s the alternative?#2020-06-2617:59pmonks(let me move this to #off-topic)#2020-06-2720:33practicalliI published a page on using the very unofficial tools.deps hot-loading of dependencies. Be aware that this is very unofficial and no guarantee it will be included in anything official. Credit to Sean Corfield for the alias configuration. I've shown an example of using hot loading with the rebel REPL. https://practicalli.github.io/clojure/alternative-tools/clojure-tools/hot-load-dependencies.html
Again, this is very unofficial, so use with caution and dont rely on it being available.#2020-06-2720:45alexmillerGiven that this is an unreleased branch of an alpha lib and we’ve already determined the api is going to change, can you make it real obvious on that page that this is not “official” stuff?#2020-06-2720:46alexmillerThere is a wip add-lib2 branch out there but it is actually a branch off another wip branch (which has now moved forward and not been merged back)#2020-06-2720:48alexmillerIt’s entirely possible for that existing add-lib branch to become broken at any time (as master drifts), and I actually expect that to happen soon#2020-06-2720:54seancorfield@alexmiller Understood. I rely on it quite a bit but I'm happy to switch to any other working version, if I can just prevail on you to point me at such a thing if the SHA I'm depending on right now breaks (and I'm happy to use Clojure master for dev, if that's what I have to switch to)#2020-06-2721:02alexmillerThere may be a period of time where there is no working add-lib branch#2020-06-2721:03seancorfieldI haz a sad 😢#2020-06-2721:03seancorfieldFair enough. I can survive for a "period of time" without it I guess.#2020-06-2721:05alexmillerI will try to get it remerged, just won’t be my priority so may take a few days as the tools.build stuff moves forward#2020-06-2721:05seancorfieldCool. And I'm happy to test that out too, as you know 🙂#2020-06-2720:55seancorfield^ @jr0cket I would have tried to dissuade you from writing this up, if you had asked me ahead of time. Even tho' it's in my public dot-clojure file and I use it day-in, day-out, I wouldn't want to see that baked into the public documentation base around Clojure...#2020-06-2720:58practicalliI did add a caveat to that effect in the deps.edn file, however, I should have put that same warning on the page too. I will update asap#2020-06-2721:04seancorfield@jr0cket This is why I've held off doing a YouTube video showing how I use add-lib on that t.d.a branch -- I don't think people should be encouraged to use it when it's likely to go away/break/change.#2020-06-2721:09practicalli@alexmiller I've updated the page with a large up front warning. If you prefer I remove the page entirely, I am happy to do so.
I appreciate there are other approaches and I should probably work on documentation for them in hindsight, its just someone was asking me how to specifically do this hot reloading.#2020-06-2721:16alexmillerIt’s a public thing, so I don’t care if people are aware of it, just want to be sure it’s experimental nature is clear#2020-06-2812:33practicalliI've moved the page on add-lib to a reference section of the book, as an alternative and very much experimental approach. Apologies for not thinking about this better.#2020-06-2810:32borkdudeWhat does this mean?
Exception in thread "main" clojure.lang.ExceptionInfo: Manifest type not detected when finding deps for babashka.curl/babashka.curl in coordinate {:git/url "", :sha "2dc7f53271de3c2edc3e1474a000b9dfa7324eaf"} {:lib babashka.curl/babashka.curl, :coord {:git/url "", :sha "2dc7f53271de3c2edc3e1474a000b9dfa7324eaf"}}, compiling:(/tmp/prog.clj:3:1)#2020-06-2810:32borkdudeas a result of:
(require '[clojure.tools.deps.alpha :as deps])
(-> {:deps '{babashka.curl
{:git/url ""
:sha "2dc7f53271de3c2edc3e1474a000b9dfa7324eaf"}}
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}}
(deps/resolve-deps nil)
(deps/make-classpath nil nil))#2020-06-2810:40borkdudeI get the same result when using this coordinate with -Sdeps from the command line#2020-06-2810:59lreaddue to empty :deps? https://github.com/borkdude/babashka.curl/blob/2dc7f53271de3c2edc3e1474a000b9dfa7324eaf/deps.edn#L1#2020-06-2812:42alexmillerthis should not affect anything#2020-06-2813:47lreadoh, looks like I misdiagnosed! thanks for the info!#2020-06-2813:48alexmillerit's just map merges and stuff in the code, so I would not expect it to violate any Clojure-ish expectations here#2020-06-2811:02borkdudeThat appears to be it. Weird.#2020-06-2811:03borkdudeThanks!#2020-06-2811:05lreadYou are most welcome, by chance I hit that one yesterday.#2020-06-2812:43alexmillerdo you have a repro of that?#2020-06-2811:07borkdudehttps://clojure.atlassian.net/browse/TDEPS-156#2020-06-2811:26borkdude@lee Hmm, now it's coming back:
$ clj -Sdeps "{:deps {babashka.curl
{:git/url \"\"
:sha \"cbf8429addac1d2f71ac8ddfb847657223fdf0f7\"}}}" -Spath
Error building classpath. Manifest type not detected when finding deps for babashka.curl/babashka.curl in coordinate {:git/url "", :sha "cbf8429addac1d2f71ac8ddfb847657223fdf0f7"}
#2020-06-2811:28borkdudeit did work for:
$ clj -Sdeps "{:deps {babashka.curl
{:git/url \"\"
:sha \"1dc0f61d7baabee1f009543d60bfb79ee4a7101c\"}}}" -Spath#2020-06-2811:29borkdudewhile the commit cbf8 didn't do anything relevant compared to 1dc0with the deps.edn file....#2020-06-2811:31borkdudeAdding :deps/manifest :deps seems to fix it.#2020-06-2811:33borkdudeAdded this to the issue#2020-06-2811:53borkdudeI'm also getting this for a local/root dep:
$ clj -Sdeps "{:deps {org.clojure/tda {:local/root \"/Users/borkdude/Dropbox/dev/temp/tools.deps.alpha\" }}}" -Spath
Error building classpath. Manifest type not detected when finding deps for org.clojure/tda in coordinate #:local{:root "/Users/borkdude/Dropbox/dev/temp/tools.deps.alpha"}#2020-06-2812:03alexmillerAre they reproducible with -Sforce#2020-06-2812:04borkdudesame result#2020-06-2812:05borkdude/tmp $ clj -Sdeps "{:deps {babashka.curl
{:git/url \"\"
:sha \"cbf8429addac1d2f71ac8ddfb847657223fdf0f7\"}}}" -Spath -Sforce -Srepro
Error building classpath. Manifest type not detected when finding deps for babashka.curl/babashka.curl in coordinate {:git/url "", :sha "cbf8429addac1d2f71ac8ddfb847657223fdf0f7"}#2020-06-2812:38alexmiller@borkdude I can't repro any of your failing cases#2020-06-2812:39borkdudeHah. Worked after a rm -rf ~/.gitlibs#2020-06-2812:39borkdudeNot for
clj -Sdeps "{:deps {org.clojure/tda {:local/root \"/Users/borkdude/Dropbox/dev/temp/tools.deps.alpha\" }}}" -Spath
though#2020-06-2812:41borkdudeThat's probably another problem. It does work with a vanilla checkout of tools.deps:
clj -Sdeps "{:deps {org.clojure/tda {:local/root \"tools.deps.alpha\" }}}" -Spath#2020-06-2812:41borkdudeI'll close the issue.#2020-06-2812:44alexmillertools.deps.alpha has a couple build steps (java compilation, resource replacement) so it's not a good example lib for use as local/git dep#2020-06-2812:50alexmillerbtw, in the near future, using unqualified lib symbols is going to be deprecated and will complain, so "babashka.curl" should be changed to "babashka.curl/babashka.curl" etc#2020-06-2812:51alexmiller(and this is a step towards actually removing support for that)#2020-06-2813:24borkdude@alexmiller I know. I had already compiled the java and added target/classes to the paths#2020-06-2813:24borkdudeBut still didn’t work probably due to the resource replacement #2020-06-2813:26borkdudeI’m trying to get the native tools deps working using Ghadi’s git shell branch of the git libs library, but this stuff got me distracted for now #2020-06-2813:30borkdudeShould probably push the compiled assets to Clojars and work with that instead of tools deps deps #2020-06-2813:36borkdudeBtw, the url of a git dep is already a unique identifier. Why should a user have to make up some fully symbol keyword in the first place?#2020-06-2813:41alexmillerwell you can just mvn clean install and use that local snapshot artifact too - sometimes that's eaesiest for maven built things#2020-06-2813:47alexmillerre unique id, for the exact same reasons namespaces are useful for global differentiation, group-ids are a good idea for qualified names. using same symbol group+artifact was always a limited idea. lein/clojars made it too easy and common. we're not going to perpetuate making it implicit.#2020-06-2813:52borkdudeBut why have a name for a git dep at all? #2020-06-2813:54alexmillerbecause names are the identity of the lib, which may exist at different coordinates and you want to avoid putting the "same" lib on the classpath from multiple locations#2020-06-2813:55alexmiller(admittedly still some work to do on cross-coordinate type version selection, but the hooks are there)#2020-06-2813:57borkdudeGood point. The only use case for doing that would be when you would mount different dirs from the same git dep at different SHAs, which seems far fetched. Still if you would use the URL as the identifier you would be able to detect conflicts more easily since different people are going to use different qualified syms for the same lib?#2020-06-2908:07rickmoynihanNot that far fetched as we actually do this in one or two places.
One of our apps supports different customer configurations via t.d.a aliases, and there is essentially a :customer/all alias that pulls together a combined classpath for all customers, for working at the REPL (which lets you switch between customers).
One of the deps in that alias contains fixture data for all customers with a :deps/root for each customer, and it’s possible for the sha’s for customers to drift slightly from each other (even though they’re all in the same repo). So this t.d.a feature is very handy, as otherwise we’d need a fixture data repo per customer, which would make updating fixture harder across customers harder.
Essentially it just means the artifact id in this becomes git url + deps-root. It makes a lot of sense when you use :deps/root.
Normally they all reference the same SHA, but sometimes it’s pragmatic to apply changes to one customer first, before rolling out wider. This can also happen when you want a branch to built repeatably by C.I. (before you’ve applied the change across all customers).#2020-06-2908:17rickmoynihanWe then coin a local dep-id of something like: swirrl/fixtures.customer-name#2020-06-2908:19borkdudeYeah, that does make sense.#2020-06-2908:21rickmoynihanIt’s essentially just a lighter weight way to manage separate dependencies that you often make cross-cutting changes over.#2020-06-2908:24rickmoynihanThough I do suspect t.d.a might have some bugs resolving these sometimes — I’ve seen a few cases where when fetching these deps it seems to either blow up or mix up the commits, e.g. it would resolve customer-b to customer-a’s sha.
I suspect it might be related to the parallel resolution stuff. cc @alexmiller#2020-06-2908:24rickmoynihanIt’s quite hard to reproduce though#2020-06-2913:09alexmilleryes, I am almost certain that it's parallel resolution of git deps#2020-06-2813:58alexmillerwe do not currently do that level of detection but it is something I would like to do#2020-06-2813:59alexmillerultimately the library has to determine the identifier by which it should be known#2020-06-2814:01alexmillerthe organization should be something you own or register, like domain name, trademark. third party sites like github can also confer identity - we recommend in https://clojure.org/guides/deps_and_cli to combine them by creating a group-id like github-borkdude#2020-06-2814:01borkdudeWhich the URL is already an instance of #2020-06-2814:04alexmillerurl is really a combination of site (github), org, and repo. github-org/repo combines those#2020-06-2814:04alexmillerbut it does not account for forks#2020-06-2814:04borkdudeInteresting 🤔 #2020-06-2814:05alexmillerto use a fork, you need to use the same lib, but different url, and then they diverge#2020-06-2814:05alexmilleryour goal with using a fork really is to replace (identity) the lib#2020-06-2814:06alexmillerbut then you have interesting questions of ownership over the lib identity#2020-06-2814:06borkdudeIf I want to maintain a fork as a real fork, and not just for PRs, I would not use the original identifier anyway? #2020-06-2814:06alexmillerwell that's the question#2020-06-2814:06borkdudeSo then the URL still is the most identifying thing for it #2020-06-2814:07alexmillerif you didn't use the original, then dep resolution will give you both libs if you have a transitive dep using the original#2020-06-2814:07alexmillerand in that case, it would be good to have deps tell you that you've done this#2020-06-2814:07borkdudeIt’s the same in maven world right now #2020-06-2814:07alexmillertrue#2020-06-2814:08alexmilleralthough probably much harder to automatically detect#2020-06-2814:09alexmiller(note that git urls are also not unique - there are multiple url formats to refer to the same repo, but I think that's a solvable problem)#2020-06-2816:27borkdudeI have got something compiling now with tda + /cdn-cgi/l/email-protection branch
One issue I stumbled upon:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: org.apache.maven.project.ReactorModelPool. To diagnose the issue you can use the --allow-incomplete-classpath option. The missing type is then reported at run time when it is accessed the first time.
Detailed message:
Trace:
at parsing clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(pom.clj:51)
Call path from entry point to clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(Object):
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(pom.clj:41)
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invoke(pom.clj:41)
at clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.hashCode(Unknown Source)
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at java.util.Properties.getProperty(Properties.java:1125)
at com.oracle.svm.core.jdk.SystemPropertiesSupport.getProperty(SystemPropertiesSupport.java:144)
at com.oracle.svm.core.jdk.Target_java_lang_System.getProperty(JavaLangSubstitutions.java:345)
at com.oracle.svm.jni.JNIJavaCallWrappers.jniInvoke_VA_LIST:Ljava_lang_System_2_0002egetProperty_00028Ljava_lang_String_2_00029Ljava_lang_String_2(generated:0)
I now use --allow-incomplete-classpath just to see how far it gets.
I now get a NPE in util/maven.clj on the line local-repo-mgr (.newLocalRepositoryManager system session local-repo) where system appears to be null. Might be related to the --allow-incomplete-classpath but I'll keep digging.#2020-06-2816:29alexmillerIt is almost certainly related#2020-06-2816:30alexmillerThere is a similar thing like this logged where an error occurs while setting up the service locator#2020-06-2816:38borkdudeSo ditching jgit for shell git works. When are you going to ditch org.apache.maven libs? 😉#2020-06-2816:47borkdudeI wonder what's the issue with this class#2020-06-2817:19borkdudePosted current findings + code here: https://github.com/borkdude/tools-deps-native-experiment#2020-06-2818:49ghadibefore using a solution like shelling out to git, I think it’s wise to re-examine the problem#2020-06-2818:49ghadiI know I sound like a broken record#2020-06-2818:56borkdudeAlex pointed me at that shell-git branch when I told him jgit didn't work with GraalVM. Just examining + listing all the problems. Shelling out to git allows me to explore the next problem without being stuck on it. There might not be only one "the problem", there might be many.#2020-06-2819:07ghadiwhat part of jgit didn't work?#2020-06-2819:08borkdudeI think the output I got was similar to this one: https://bugs.eclipse.org/bugs/show_bug.cgi?id=546175#2020-06-2819:09borkdudeI think that problem should be solvable using the approach quarkus takes: https://quarkus.io/guides/jgit#2020-06-2819:09borkdudeso if there's need, it can be made to work, I'm pretty sure#2020-06-2819:11borkdudeI added this as a note to the README here now https://github.com/borkdude/tools-deps-native-experiment/blob/master/README.md#2020-06-2908:49holyjakHello! Could somebody be so kind and explain to me why clj is trying to download a dependency from Maven Central when I told it to you our company repository (I assume M.C. is always used as a back up; but it doesn't seem it has even tried?!:
$ cat deps.edn
{:mvn/repos {"telia-aws" {:url ""}}
:deps {com.oracle.jdbc/ojdbc8 {:mvn/version "12.2.0.1"}}}
$ deps.exe -Sverbose
deps.clj version = 0.0.8
...
Refreshing classpath
Error building classpath. Could not find artifact com.oracle.jdbc:ojdbc8:jar:12.2.0.1 in central ()
🙏 Thank you!#2020-06-2908:51borkdudeI think that error message just picks a repo and doesn't list all of them#2020-06-2910:45holyjakIt seems you were right! The problem turned out to be that my URL was wrong, it was missing a part of the path. The -Sverbose option was quite useless for troubleshooting this... :-(#2020-06-2911:13holyjakAny idea why is clj trying to download a snapshot .jar from Central (where it isn't) when I have it in the local .m2 repo? Do I need to tell it to use the local repo??
> Error building classpath. Could not find artifact no.telia:clj-common:jar:0.1.0-20200507.122109-9 in central (https://repo1.maven.org/maven2/)
ls ~/.m2/repository/no/telia/clj-common/0.1.0-SNAPSHOT/clj-common-0.1.0-20200507.122109-9.jar
/Users/me/.m2/repository/no/telia/clj-common/0.1.0-SNAPSHOT/clj-common-0.1.0-20200507.122109-9.jar
The dependency is declared as:
no.telia/clj-common {:mvn/version "0.1.0-20200507.122109-9"} ;; a particular SNAPSHOT
#2020-06-2911:16borkdudeSNAPSHOT versions are refreshed from external repositories at least once a day (by default, you can configure this in .m2 settings I believe)#2020-06-2911:16borkdudeit's maven behavior#2020-06-2911:20holyjakAh, ok. And if it fails to refresh, it just crashes without using the old version?#2020-06-2911:21borkdudehttps://stackoverflow.com/questions/1348603/how-can-i-get-maven-to-stop-attempting-to-check-for-updates-for-artifacts-from-a#2020-06-2911:21holyjakI found https://maven.apache.org/ref/3.6.2/maven-settings/settings.html#class_snapshots, I can try to set updatePolicy to never...#2020-06-2913:25alexmillerdeps doesn't currently expose this setting#2020-06-2915:51holyjakSo it doesn't take all of settings.xml into account, only parts of it (ie auth)?#2020-06-2915:52alexmillerit does not consider that setting, it does take more than just auth though#2020-06-2915:52alexmillerhttps://clojure.org/reference/deps_and_cli describes supported stuff#2020-06-2915:53alexmillerspecifically, also supports mirrors and proxies#2020-06-2915:53alexmillerand http properties#2020-06-2911:32holyjakHm, it seems that clj simply ignores my local .m2/repository. Now it is trying to download com.oracle.jdbc:ucp:pom:12.2.0.1 even though ~/.m2/repository/com/oracle/jdbc/ucp/12.2.0.1/ucp-12.2.0.1.jar exists...:
$ cat deps.edn
{:deps {com.oracle.jdbc/ucp {:mvn/version "12.2.0.1"}}}
$ clj
Error building classpath. Could not find artifact com.oracle.jdbc:ucp:jar:12.2.0.1 in central ()
$ ls -lh ~/.m2/repository/com/oracle/jdbc/ucp/12.2.0.1/ucp-12.2.0.1.jar
-rw-r--r-- 1 holyjak staff 1.0M Apr 2 13:55 /Users/holyjak/.m2/repository/com/oracle/jdbc/ucp/12.2.0.1/ucp-12.2.0.1.jar
Why?!#2020-06-2913:23alexmillerdid you put that jar manually in your .m2?#2020-06-2913:24alexmillerif so, you may be missing some metadata in your local cache that's triggering a lookup#2020-06-2915:52holyjakNo, via mvn install-file . And it used to work...#2020-06-2915:56alexmillerbased on what I see here, I would expect it to find and use the jar in your local cache#2020-06-2915:56alexmilleryou might want to clj -Sforce to make sure you don't have a cached classpath or something#2020-06-2915:57alexmillerI don't think that will matter since it seems to be building the classpath when it encounters the error, but just a good habit when diagnosing clj stuff#2020-06-2920:43holyjakOk, thank you!#2020-06-3011:22holyjakFYI my problem is fixed now. I believe something was wrong with my .m2/repository#2020-06-3012:01alexmillerCool#2020-06-3015:46unbalancedHeya @alexmiller I'm reading through tools.deps and you're using some pretty advanced kung fu in there. I don't suppose in all your ample free time you've given any talks or write-ups on a deep dive there have you? I'm trying to write some clojurescript CLI tools for working with deps and trying to port some of the stuff. Also, I tried running with CIDER but nREPL on the deps ran into the predictable errors for including java classpaths -- I'm curious what your REPL technique is for developing tools.deps.alpha?#2020-06-3016:44alexmillerThe last conj talk is probably the most detailed info about the impl. What part are you having trouble with? Re repl, I just open it in Cursive as a Maven project#2020-06-3015:47borkdude@goomba There are a couple of tools.deps talks, they are listed in the repo#2020-06-3015:48borkdudeHmm, only the one from EuroClojure is https://www.youtube.com/watch?v=sStlTye-Kjk. But there's another one from the latest Conj.#2020-06-3015:48unbalancedI was there for the last one. I think I'm looking for more "how to contribute" style info#2020-06-3015:49unbalanced😄 reading seems like a good start#2020-06-3015:49borkdude@goomba ClojureScript already has cljs.main which works great in tandem with tools.deps?#2020-06-3015:50unbalancedyeah of course. I'm not trying to replicate that functionality -- I'm working on something more like "pip for deps"#2020-06-3015:50unbalancedI'm lazy and I don't like editing my deps file by hand#2020-06-3015:50unbalancedalso I can never remember version numbers#2020-06-3015:50unbalancedI've got it working for clojars but there's a bunch of stuff on git and maven and I don't know how to work with those yet#2020-06-3015:51borkdudeinteresting, anything public yet?#2020-06-3015:51unbalancednot yet because it's still a mess but I'll send you some stuff on what I've got#2020-06-3015:53dpsuttonthere is a project out there that will search and add things to your deps files#2020-06-3015:54dpsuttonhttps://github.com/hagmonk/find-deps#2020-06-3015:57unbalancedAhh that's great. Reason I'm going the cljs route is just b/c of startup time etc#2020-06-3015:58unbalancedand for personal education, I suppose#2020-06-3015:58borkdude@goomba if startup time is important, you can also use GraalVM and/or babashka#2020-06-3015:58unbalancedI need to learn to use both of those, for sure#2020-06-3015:58borkdude(all good options, including what you're doing now of course!)#2020-06-3015:58dpsuttondidn't mean to derail your exercise. but you can peruse the good if you like 🙂#2020-06-3015:58unbalancedI'm hear here to learn in all forms!#2020-06-3015:59unbalancedHas GraalVM gotten to the point where we can write arbitrary CLI stuff in Clojure now?#2020-06-3015:59unbalanced(out of the loop)#2020-06-3015:59borkdudeyeah, but you can do a lot with babashka as well, without compiling.#2020-06-3016:00unbalancedI need to sneak that one in at work 😈#2020-06-3016:00borkdude@goomba meta-example: https://github.com/borkdude/deps.clj
that is the clojure bash script ported to both babashka and GraalVM, so you can either run it as script or binary#2020-06-3016:01unbalancedyou can even babashka in powershell!? @borkdude your level of masochism is very impressive#2020-06-3016:01borkdudeyeah, Windows supported too#2020-06-3016:02unbalancedsome heroes don't wear capes#2020-06-3016:02borkdudehow do you know I'm not wearing a cape? (haha, just kidding...)#2020-06-3016:03unbalancedwell with covid I'm willing to wager most aren't even wearing pants. So I was just playing the odds#2020-06-3016:04unbalancedIt looks like find-deps , from a quick glance, is dealing with the same limitation I have#2020-06-3016:05unbalancedwhich is finding reagent is easy but org.clojure/core.match or org.clojure/core.async is not as easy#2020-06-3016:05unbalancedI'm not sure how to differentiate in Maven what is Clojure and what is Java, for instance, conceptually#2020-06-3016:06unbalancedor... maybe it doesn't matter -- as I say it out loud. That's kind of the point of Clojure, isn't it? 😆#2020-06-3016:17dpsuttonthere are some nice todos and vision in that find-deps lib that you could tackle#2020-06-3016:17dpsuttondealing with rankings and preferences if i recall correclty#2020-06-3016:47alexmillerIf you’re talking about searching/finding deps, we’ve actually done some thinking about this in tandem with add-lib and I’ve even prototyped some of it in the add-lib2 branch of tools.deps but it’s not in a usable state right now#2020-06-3016:48alexmillerI guess really more in the context of repl dev not updating deps.edn#2020-06-3016:49alexmillerBut could potentially be used for either I guess#2020-06-3022:44flefikIs there a good commit on the add-lib or add-lib2 branch that can be checked out and played around with?
I'm thinking of adding a filewatcher on deps.edn and just automatically add any new libraries that are added there without having to restart the repl.#2020-06-3023:29seancorfieldI have an example in my dot-clojure repo that I use all the time -- but @alexmiller has cautioned that add-lib will probably break as code moves around#2020-06-3023:29seancorfield^ @UAEFFG05B#2020-07-0107:47flefikthanks!#2020-06-3022:49lreadhey @aviflax did you ever get a tools dep answer for tar.gz files ending up on your classpath? https://github.com/oracle/graal/issues/2481#2020-06-3022:50aviNo#2020-06-3022:50aviI meant to ask Alex about it here… thanks for the reminder#2020-06-3022:51aviThe comment on that issue from olpaw to the effect that the tar.gz file shouldn’t have been included in the classpath in the first place made sense to me.#2020-06-3022:51aviSo I thought maybe this might be one of those places wherein tools.deps is deviating from commonly assumed Maven behavior/convention#2020-06-3022:52aviI don’t really know ¯\(ツ)/¯#2020-06-3022:54lreadYeah, I bet @alexmiller knows! Here’s the dep that brings in tar.gz files: https://github.com/FundingCircle/repro-graalvm-mac-issue/blob/83abc805467808221cf48108f735527e3e3de9fa/deps.edn#L3#2020-06-3022:56lreadI guess this does not happen with leiningen, as @borkdude references it from babashka project.clj https://github.com/borkdude/babashka/blob/e667bb5d2f344fdfd12ce12639c768d80e252138/project.clj#L57#2020-06-3022:58aviRight#2020-06-3023:00aviAs I mentioned in that graal issue, the tar.gz file comes from https://search.maven.org/artifact/org.graalvm.truffle/truffle-nfi/20.1.0/jar for org.graalvm.truffle/truffle-nfi 20.1.0:
<dependency>
<groupId>org.graalvm.truffle</groupId>
<artifactId>truffle-nfi-native-linux-amd64</artifactId>
<version>20.1.0</version>
<type>tar.gz</type>
</dependency>#2020-06-3023:01aviI assume must be some use case for including such a file in a pom, although I have no idea what it is. And I assume that Maven, and whatever mechanism Leiningen uses to work with Maven, somehow filters these files out when/where they’re not needed/useful.#2020-06-3023:23alexmillerThat’s a native dep and there is a facility in Maven to support loading those for the jvm#2020-06-3023:24alexmillerThat Linux and amd64 there are actually parsed and used to select the right native dep for the os/arch#2020-06-3023:25alexmillerI think lein supports at least some of this too, but deps does not#2020-06-3023:26lreadThanks @alexmiller!#2020-06-3023:34lread@aviflax I have something working for my project, I’ll head over to #graalvm to followup#2020-06-3023:34aviGreat!#2020-07-0111:38practicalliDoes the clojure or clj command from Clojure CLI tools read a user.clj file if it contains (ns user) expression when starting the REPL?
With Leiningen projets using a user or dev/user.clj file was one way to start component services.
For a Clojure CLI project, if the dev directory was added as an extra path under a :dev alias and clj -A:dev used to start the REPL, should the dev/user.clj file be automatically evaluate on REPL start?
I though I had this working the other week, but not sure if I imagined this now 🙂#2020-07-0111:47borkdude@jr0cket I think the answer is yes, it's not tooling dependent: https://github.com/clojure/clojure/blob/d07ef175c700329f7afbef8770332b6247a34a49/src/jvm/clojure/lang/RT.java#L486#2020-07-0111:54practicalliAh, I have it working in terms of requiring a namespace and evaluating a funtion call. However, using (in-ns main-project-namespace) does not seem to switch to the namespace (or if it does, then maybe it been switched back to user afterward).#2020-07-0111:57borkdude@jr0cket load-file also switches back to the original namespace afterwards, because it uses binding + *ns*#2020-07-0111:57borkdudeI imagine something similar is happening here#2020-07-0112:13practicalliOkay, I dont have to change the namespace, I can require the relevant libraries and use fully qualified namespace function calls.
https://practicalli.github.io/clojure/clojure-tools/configure-repl-startup.html
If there is a way in Clojure CLI tools to change the namespace using a user.clj file, I would be interested to know, but not essential. I can use the --eval option to run the relevant expressions.#2020-07-0112:15borkdude@jr0cket This seems to work:
$ clj -e "(ns foo.bar) (alter-var-root #'*ns* (constantly 'foo.bar))" -r
#2020-07-0112:17borkdude@jr0cket or even:
$ clj -e "(ns foo.bar) (alter-var-root #'*ns* (constantly 'foo.bar)) (clojure.main/repl)"
foo.bar
foo.bar=>
#2020-07-0112:18practicalliI can set the namespace easily using --eval option on the command line. Unfortunately the above do not work from within user.clj. Ideally I'd like use it with rebel readline, which is the REPL UI I use if I am just using the REPL directly rather than an editor. --eval doesnt work when using the :rebel alias#2020-07-0112:22borkdude@jr0cket Very well.
$ clj -R:rebel -e "(ns foo.bar) (alter-var-root #'*ns* (constantly (find-ns 'foo.bar)))" -m rebel-readline.main
#object[clojure.lang.Namespace 0x46cf05f7 "foo.bar"]
[Rebel readline] Type :repl/help for online help info
foo.bar=>
#2020-07-0112:23practicalliSuccess 🙂 I have learnt many things today, thank you.#2020-07-0119:52drewverleeIs there a way to do a universal exlucions? i see you can attach exluded to individual deps, but what if i wont want this lib from any dep?#2020-07-0119:55borkdude@drewverlee you can use :classpath-overrides:
{:aliases {:babashka {:classpath-overrides {org.clojure/clojure nil
org.clojure/spec.alpha nil
org.clojure/core.specs.alpha nil}}}}#2020-07-0119:57borkdudeThis only works in an alias#2020-07-0119:58drewverleegotcha.#2020-07-0213:22arohneris there a library that provides lein deps :pedantic :abort? i.e. looks for and warns on conflicting versions?#2020-07-0213:23arohneror documentation on how conflicting versions are handled?#2020-07-0213:27delaguardoyou could use -Strace to create edn file with all that information. but I’m not aware of any tooling to make reading of that file easy#2020-07-0213:28alexmillerthere are a few tools like depot#2020-07-0213:28alexmillerhttps://github.com/clojure/tools.deps.alpha/wiki/Tools#2020-07-0213:29alexmilleror https://github.com/slipset/deps-ancient#2020-07-0213:30arohnerI’m looking to handle a jarhell situation where I declare dependencies on A and B, and they each declare a dependency on C, but with conflicting versions#2020-07-0213:31sveriYou could refernce the conflicting dependency as a top level dependency and decide for a version yourself#2020-07-0213:32arohnerRight, but I need to discover conflicts before resolving them. I’d like a tool to detect them and warn before I find out the hard way#2020-07-0213:30arohner:pedantic :abort warns about that, in (some) situations#2020-07-0213:31arohnerbut I’d also need to understand how tools.deps resolves that tree. lein’s dependencies are a vector, so deps are ordered sequentially. With a map, how does that work?#2020-07-0213:33alexmillerwell I did whole conj talk about it. it's complicated#2020-07-0213:34arohnerIs the talk uploaded?#2020-07-0213:36alexmilleryeah - https://www.youtube.com/watch?v=7CM7Ef-dPWQ#2020-07-0213:33alexmillerbut generally the idea is to pick the newest version#2020-07-0213:34alexmillerbut it also picks consistent subtrees (so no child lib should be included if it's parent isn't)#2020-07-0213:35alexmillerclj -Strace is the best info about what it did and why - the log details each lib/version it looked at (breadth first), what it decided, and why#2020-07-0213:36alexmillertools.deps.graph has a tool for using trace.edn files to generate a series of pictures from that that can be somewhat helpful as well#2020-07-0216:38dominicmMy argument is that pedantic abort doesn't make sense in a deps world.#2020-07-0216:40dominicmYou'll never get an older version than what is required, which eliminates 90% of problems you get with maven. In maven, it's assumed everything is conflicting. Deps assumes that newer things are compatible with older things (so the conflict is picking the older dependency?).#2020-07-0216:42dominicmHowever, the equivalent tool I think would be useful is detecting a top level dependency on an older artefact, when you've bumped the version and it transitively depends on something newer.#2020-07-0219:41markbastianIf I have a dependency specified in my main deps section and another version of that dependency under an alias, does the alias take precedence when it is active?#2020-07-0219:43markbastianAh, think I found it at https://clojure.org/guides/deps_and_cli. Just use override-deps.#2020-07-0220:15alexmilleryes#2020-07-0221:04markbastianIs there a way to specify a subdirectory when providing git coordinates for a dependency? For example, the git project is foo and within that you have the folders foo-clj and foo-py. deps.edn is in foo-clj. I want to point deps to the foo-clj folder.#2020-07-0221:06borkdude@markbastian yes, :deps/root#2020-07-0315:40arohneris there a way to exclude a dep from the tree, regardless of how it got there?#2020-07-0315:40arohner:exclusions appears to be per-dep only, but I want to assert e.g. logback should never be present#2020-07-0315:49borkdude@arohner https://clojurians.slack.com/archives/C6QH853H8/p1593633340186100#2020-07-0319:31neumannIs there a way to replace or remove paths in an alias? I want to exclude on of my source paths in just one alias.#2020-07-0319:32neumannMy current coping mechanism is to set -Sdeps '{:paths ["some" "paths" ...]}' on the command line.#2020-07-0319:45dominicm@neumann why not move the optional paths to an alias?#2020-07-0319:50neumann@dominicm I'd like to avoid that because I don't want to have to specify another -A: in all of the cases except one. The extra -A: is just another thing to forget to do.#2020-07-0319:54borkdude@neumann major hack, this removes the src path from the classpath:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "b3d1dcc1d8d7c6d7d6f3c5c3c0828a828b"}, :content ("[email protected]")}#2020-07-0319:56neumann@borkdude Nice! Yes, I was thinking of maybe using babashka. (Which is amazing! Thank you so much for creating it!)#2020-07-0320:01neumann@borkdude Something like this, using "src" as the example:
cat deps.edn | bb -I -e '(->> *input* first :paths (remove #{"src"}) vec)'#2020-07-0320:02borkdude@neumann if you need only one EDN input value:
$ cat deps.edn | bb -e '(->> *input* :paths (remove #{"src"}) vec)'
["parser" "resources" "inlined"]#2020-07-0320:02borkdude-I is for collecting a lazy seq of multiple EDN input values#2020-07-0320:03neumannOh! Very nice! Thanks!#2020-07-0418:28p-himikIf A depends on B, and my project depends on A and B but doesn't really care about the version of B as long as it's compatible with A, how should I specify this dependency on B in deps.edn?
Or should I just omit the dependency on B from deps.edn?#2020-07-0419:10seancorfieldI would figure out what version of B your chosen version of A depends on and explicitly specify that in deps.edn -- and add a comment about the version selection reason. Assuming you mean that your project has an explicit dependency on B.#2020-07-0419:23p-himikThat was exactly my thought initially. But now that I've given it some thought - wouldn't it be more practical to just omit it altogether?
If A starts to depend on a different version of B, I should be unaffected, and I won't need to change anything.
If A drops the dependency, the code will just not run because there's an explicit :require.
And specifying B in my deps.edn seems like just giving myself more work when I update my dependencies.#2020-07-0419:33seancorfieldIf A changes the version of B it depends, it also changes its own version -- so unless you are changing the version of A you depend on, you wouldn't be affected 🙂#2020-07-0419:34seancorfieldThat's my point: if you choose to depend on a newer version of A, you should also verify the compatibility of the newer version of B that it now depends on as well. Having an explicit dependency on B lets you force the earlier version of B if you need that.#2020-07-0419:35seancorfieldIf A stops depending on B, that also means a new version of A and you could decide to stay with the old version or move to the new version -- and because you have an explicit dependency on B, you code won't break (at least, not because A's transitive dependency on B went away).#2020-07-0419:36seancorfieldI really don't like relying on transitive dependencies. I much prefer to have all my dependencies listed explicitly. I've learned that over the years after a decade of Clojure in production 🙂#2020-07-0419:43p-himikHmm, I think I didn't give enough context.
I definitely agree with your reasoning about the generic case of relying on a transitive dependency. But in my particular case, the code should either break completely if B is changed/removed, or it should continue working just as I expected it to. So the compatibility with my project is not a concern.
At the same time, if I manage to forget to update B after updating A, I may introduce some incompatibilities between A and B that are hard to test.#2020-07-0419:44p-himikTo be more concrete - I'm just using ring.util.mime-type/ext-mime-type function without having an explicit dependency on ring/ring-core.#2020-07-0419:46p-himikOf course even such a dead simple function can potentially be broken with an update. But I consider it so improbable that it's not worth taking into account. Akin to how I don't expect clojure.core/map to suddenly start giving me different results.#2020-07-0419:53seancorfieldBut you have code in your project that depends on Ring namespaces?#2020-07-0419:54seancorfieldTo be concrete, what are A and B specifically in your project?#2020-07-0420:02p-himikA - yada/core
B - ring/ring-core
I need only ring.util.mime-type/ext-mime-type from B, which I consider stable in a sense that any changes around it either break the app startup or don't affect the app at all.#2020-07-0420:02p-himikI don't depend on anything ring-related but that small function.#2020-07-0420:08seancorfieldAh, OK. If that is the only Ring namespace you're touching then maybe I would agree and just depend on Yada. It still feels a bit sketchy to me. We depend on Ring and we don't use that var -- we have our own MIME type structures (because we want a specific set of supported values, rather than a generic list).#2020-07-0420:13p-himikI wish I would have a specific set. :)
Yeah, I can see how it can feel sketchy. I'm still not sure about the way I work with my dependencies at all right now, given that there's also CLJS and a build process that involves shadow-cljs. Maybe this all could be made nice and simple with aliases. An unrelated to the rest of the thread question - aliases cannot be grouped together under a new alias, right? So I will probably have to write a script to call clj -A:al1:al2:al3.#2020-07-0420:20seancorfieldNope. Grouping of aliases is a common request.#2020-07-0420:21p-himikI see.
Anyway, thanks for the input Sean. Much appreciated. :)#2020-07-0420:22seancorfieldIt's why at work we have a build shell script that wraps the clojure script so we can have "higher-level" aliases.#2020-07-0420:24p-himikRight, I've seen this pattern before. Sometimes even with -Sdeps, to avoid cluttering the main deps.edn I guess or to make the script more portable.#2020-07-0420:38seancorfieldOur build script serves two purposes: to allow us to run a series of clojure commands from a single invocation, and to provide various shorthands for common combinations of aliases (our aliases tend to be very granular). But we're all Clojure / backend. I suspect if we were managing ClojureScript as well, we'd be even more reliant on our wrapper script!#2020-07-0420:41seancorfieldI'm looking forward to tools.build -- I suspect it will help simplify our tool chain quite a bit.#2020-07-0420:49p-himikI don't think I've heard of it and I can't find anything that seems relevant. Is there an announcement or something like that?#2020-07-0420:59seancorfieldAlex mentioned it in an Inside Clojure blog post and has mentioned it several times here.#2020-07-0421:00p-himikHeh, found it: "tools.build - shh, top secret"#2020-07-0421:06seancorfieldGiving Cognitect's solid track record of producing simple, well-considered, composable tooling that addresses core Clojure developer needs, I'm looking forward to whatever this turns out to be 🙂#2020-07-0508:36dominicm@p-himik I've been considering writing a tool which ensures your top level dependencies are at least newer than those required by your dependencies#2020-07-0508:38p-himikThat may be useful. :+1: Although not for my problem in particular because I ended up just dropping the explicit dependency. :)#2020-07-0610:36dominicmI think that's wrong. You depend on that library.#2020-07-0610:50p-himikWe've discussed that at length with Sean just a bit above. :) It's not that wrong in my particular context, I think.#2020-07-0610:52p-himiktl;dr: I depend on the library, but I don't depend on the version of the library. And if something breaks, the app just won't start. So by not using an explicit dependency, I cannot really run time runtime errors.
But with an explicit dependency, I definitely can if I forget to update it along with other libraries that depend on it as well.#2020-07-0513:06alexmillerit would probably be pretty easy to read a trace file (or use the api and the trace data returned) to see that#2020-07-0610:36dominicmOh really? Interesting. I'd been resolving each dependency independently and comparing to the top level. Your idea is better.#2020-07-0610:37dominicmI wonder if you could build a whole lint suite out of the trace, catching other common errors (hello Jackson and jetty).#2020-07-0611:40fabraoHello all, how to listen in all interfaces for this
:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl} -Dfile.encoding=UTF-8"]}}
?#2020-07-0611:42fabraoI tried :address,10.10.20.whatever and it didn´t work#2020-07-0611:45fabrao-> Exception in thread "main" java.lang.NumberFormatException: Invalid number: 10.10.20.82#2020-07-0612:33borkdude@fabrao try escaped double quotes around the IP#2020-07-0613:09fabrao@borkdude :aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"10.10.20.82\",:accept,clojure.core.server/repl} -Dfile.encoding=UTF-8"]}} -> same error#2020-07-0613:09fabraoI tried with ' too, not working#2020-07-0614:02alexmillerthat seems weird to me. can you share the full output?#2020-07-0614:16fabraoException in thread "main" java.lang.NumberFormatException: Invalid number: 10.10.20.82
at clojure.lang.EdnReader.readNumber(EdnReader.java:224)
at clojure.lang.EdnReader.read(EdnReader.java:136)
at clojure.lang.EdnReader.readDelimitedList(EdnReader.java:766)
at clojure.lang.EdnReader$MapReader.invoke(EdnReader.java:680)
at clojure.lang.EdnReader.read(EdnReader.java:145)
at clojure.lang.EdnReader.read(EdnReader.java:111)
at clojure.lang.EdnReader.readString(EdnReader.java:67)
at clojure.edn$read_string.invokeStatic(edn.clj:46)
at clojure.edn$read_string.invokeStatic(edn.clj:37)
at clojure.core.server$parse_props$fn__8907.invoke(server.clj:152)
at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
at clojure.core.protocols$fn__8159.invokeStatic(protocols.clj:169)
at clojure.core.protocols$fn__8159.invoke(protocols.clj:124)
at clojure.core.protocols$fn__8114$G__8109__8123.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__8136.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__8136.invoke(protocols.clj:75)
at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6828)
at clojure.core.server$parse_props.invokeStatic(server.clj:146)
at clojure.core.server$start_servers.invokeStatic(server.clj:157)
at clojure.core.server$start_servers.invoke(server.clj:157)
at clojure.lang.Var.invoke(Var.java:384)
at clojure.lang.RT.doInit(RT.java:493)
at clojure.lang.RT.init(RT.java:467)
at clojure.main.main(main.java:38)#2020-07-0614:20alexmilleryou need to put that second -D in a separate string#2020-07-0614:21alexmiller{:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"10.10.20.82\",:accept,clojure.core.server/repl}" "-Dfile.encoding=UTF-8"]}}}#2020-07-0614:21fabraolet me try#2020-07-0614:22alexmillerI can't repro what you're getting#2020-07-0614:22alexmillerI get an error binding that address, but that's to be expected#2020-07-0614:26alexmillerI would expect that error w/o the quotes around the address#2020-07-0614:27borkdude@fabrao Are you doing this from a deps.edn file or from a cmd line with -Sdeps?#2020-07-0614:31fabraodeps.edn#2020-07-0614:34borkdudelinux, macOS, Windows?#2020-07-0614:39markbastianFrom what I've read, it sounds like a fairly standard approach to publishing library artifacts with deps is to create a pom with clj -S:pom and then use mvn deploy. When you do this model, are there any best practices regarding how to keep things in sync? For example, do you:
1. Regenerate the pom every time you want to cut a release? This seems wrong as you lose all of your version and other info.
2. Generate the pom once and then maintain it? This seems like a good route, but the main downside I see is keeping dependencies in sync.
I am assuming #2 is the best option. Any tips for syncing dependencies?#2020-07-0615:35seancorfield@markbastian clojure -Spom does not overwrite anything in your pom.xml except <dependencies>#2020-07-0615:48alexmillerAnd repositories and source dir#2020-07-0615:35seancorfieldAnd for publishing to Clojars, you really need a lot more in your pom.xml file than clojure -Spom provides.#2020-07-0615:38seancorfieldIf you create a new project with clj-new (typically clj -A:new lib myname/myproject) it will have a suitable pom.xml file and then for each release you just run clojure -Spom, update the version and tag in the pom for the upcoming release, commit and push those pom changes, add a release on GitHub and pull, clojure -A:jar (because clj-new makes that alias available), and then clojure -A:deploy (again, from clj-new's template)#2020-07-0615:39markbastianAh, I hadn't realized what the pom command did. I thought it generated a pom (replacing the existing). I just did an experiment based on your comment and suddenly things make a lot more sense. Very cool! Thanks for the info.#2020-07-0615:42seancorfieldPer clojure -h:
-Spom Generate (or update an existing) pom.xml with deps and paths
#2020-07-0615:40seancorfieldThe :jar alias is for depstar and it gets the group, artifact, and version from pom.xml and builds everything you need inside the .jar incl. manifest, multi-release, etc.#2020-07-0615:40fabrao@borkdude It´s Windows#2020-07-0615:40fabrao:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"10.10.20.82\",:accept,clojure.core.server/repl}" "-Dfile.encoding=UTF-8"]}} didn´t work#2020-07-0615:40borkdudeaaah#2020-07-0615:41seancorfieldThe :deploy alias is for deps-deploy and it expects
CLOJARS_PASSWORD=<your secret>
CLOJARS_USERNAME=<your username>
to be set in your env vars#2020-07-0615:41borkdude@fabrao windows is a bit different here maybe. can you try "" instead of \"?#2020-07-0615:41borkdudeoh no, that makes the EDN file invalid#2020-07-0615:41borkdudehmm#2020-07-0615:42borkdudetry two \"'s#2020-07-0615:43borkdudeor just try WSL2 🙂#2020-07-0615:44borkdudebut I suspect doubling the double quotes works#2020-07-0615:44fabrao\"\"10.10.20.82\"\" and ""10.10.20.82"" -> Invalid Number#2020-07-0615:45seancorfieldOr perhaps just read the t.d.a. wiki about Windows? https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#escaping-quotes#2020-07-0615:45borkdude@seancorfield he isn't using -Sdeps from the command line, this is in an EDN file#2020-07-0615:45seancorfieldI am pretty sure the same recommendations apply...#2020-07-0615:46fabraosorry ->
Error: Could not find or load main class 10.10.20.82
#2020-07-0615:46fabraoCaused by: java.lang.ClassNotFoundException: 10.10.20.82
#2020-07-0615:47seancorfield@fabrao There's a #clj-on-windows channel that has real Windows users that have probably solved this problem. You're just kind of flailing right now...#2020-07-0615:48fabraook Sean#2020-07-0615:48seancorfield(I would probably try \\" in the string in deps.edn tho')#2020-07-0615:51fabraowith \\" didn´t work too#2020-07-0615:52seancorfieldI'm firing up a Windows VM. I'll try a few years and report back in #clj-on-windows#2020-07-0616:01borkdude@fabrao, this worked for me on Windows:
{:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"\"\"127.0.0.1\"\"\",:accept,clojure.core.server/repl}" "-Dfile.encoding=UTF-8"]}}}
so doubling the escaped quotes, just like I suspected 😎#2020-07-0620:08seancorfieldTechnically that is tripling the escaped quotes#2020-07-0620:09borkdudeoh wait, I didn't even know I did that.#2020-07-0620:09seancorfieldAnd weird that it seems to work across all platforms... 😐#2020-07-0620:11borkdudehaha#2020-07-0616:01borkdudeand I verified with netstat that the server was indeed running with clojure -A:dev#2020-07-0616:19seancorfield(sorry, got distracted by a phone call with my mum in England, but I was getting there I hope...)#2020-07-0616:39seancorfieldFWIW, that format with \"\"\" around the host IP seems to work on Linux and macOS as well. Which surprised me a bit. I was able to start a REPL with that alias on Powershell, CMD (via powershell clj), and WSL -- as well as on macOS (Terminal) -- and connect to it via telnet to verify the socket REPL connection.#2020-07-0618:18cap10morganIs there a way to just install the deps w/ tools-deps? I'm looking for an equivalent to lein deps (for caching them in a Docker image build).#2020-07-0618:21borkdude@cap10morgan you can just do -Spath and will print the classpath, downloading the deps as a side effect#2020-07-0618:27seancorfield-Spath, -Stree, and -e nil seem to be the most commonly mentioned ways to have the CLI just download deps, optionally print some information about them, and just exit.#2020-07-0618:28seancorfieldI like -Stree myself since it's easier to read than -Spath if I want to verify the actual dependencies being used.#2020-07-0618:28seancorfieldOr just -e nil if I don't care.#2020-07-0618:29cap10morganGreat. Thanks!#2020-07-0619:36fabrao@borkdude It worked !!! Strange things#2020-07-0619:47borkdude@fabrao Might be good to add to the wiki.#2020-07-0623:47Spencer AppleWith the monorepo root CLJ_CONFIG deps.edn
{:aliases {:dev {:jvm-opts ["-XX:-OmitStackTraceInFastThrow"]}}}
And a local deps.edn
{:aliases {:dev {:extra-deps {...}}
I can pull configuration from the monorepo root CLJ_CONFIG
CLJ_CONFIG=../.clojure clojure -A:dev
However, this doesn’t pass down the :jvm-opts from the CLJ_CONFIG deps.edn. Is this working as intended? Is there a better way to do this? (I could create another alias with this jvm-opt or perhaps use environment variables.)#2020-07-0623:48seancorfieldAliases do not compose.#2020-07-0623:49Spencer Applethere’s a line in the guide:
> If multiple maps with these keys are activated, `:jvm-opts` concatenate#2020-07-0623:49Spencer Applethat confused me#2020-07-0623:50seancorfieldRight, that means across all the aliases you specified.#2020-07-0623:50Spencer Appleah gotcha, thanks for the help!#2020-07-0623:51seancorfieldBut they have to be different aliases. I answered in the main channel how we handle this at work.#2020-07-0623:50seancorfieldYou can either have an alias in the user-level file (via CLJ_CONFIG) or in the project-level file. What we did is to have something like :dev-defaults in the user-level (monorepo) file and then :dev as an empty alias there and overridden in each project -- and then do CLJ_CONFIG=../defaults clojure -A:dev-defaults:dev#2020-07-0623:51Spencer Applegreat, this seems like a good solution!
And thanks for the quick answer 😄#2020-07-0623:52seancorfieldWe have about thirty subprojects in our monorepo (and over 100,000 lines of code overall).#2020-07-0623:53Spencer AppleThank you for sharing your deps.edn btw, I should have studied that a bit more closely…#2020-07-0623:55seancorfieldLMK if you have any Qs -- feel free to DM me any time about it.#2020-07-0715:44pmonks@seancorfield might be time to consider creating a #seans-deps-edn channel here? 😉#2020-07-0723:13pmonksIs there a preference or recommendation between these two (equivalent) styles of specifying a version:
{:deps {some/library {:mvn/version "2.7.42"}}}
and
{:deps {some/library #:mvn {:version "2.7.42"}}}
?
(yes I realise this is a stylistic question not a technical one - I’m wondering if there’s a preferred / recommended / idiomatic style for deps.edn content)#2020-07-0723:32hiredmanI'd say namespaced maps are not mentioned in https://github.com/edn-format/edn, so if deps.edn is intended to live up to the .edn on the end, leave out the namespaced maps#2020-07-0723:34pmonksOoooh that’s very interesting! Thanks!#2020-07-0800:30alexmillerthe Clojure edn reader does read namespace maps but hiredman is correct that this is not currently part of the spec#2020-07-0800:07alexmillerthe first#2020-07-0809:07holyjakHello! How do I run clj to only download dependencies for a profile and exit at once? Like this: clojure -R:nrepl -e "(println :DOWNLOADED)" ? Thanks!#2020-07-0809:39dominicm@holyjak clojure -R:nrepl -Spath should do it :)#2020-07-0810:51practicalliUsing -Spath also gives you more data should you need to debug anything (a Sean Corfield tip). I am using this form of command in my continuous integration yml configuration (in the step where I would have use lein deps with Leingingen projects)
- run: clojure -R:test:runner -Spath
https://practicalli.github.io/clojure/testing/integration-testing/circle-ci/random-clojure-function.html#2020-07-0811:00holyjakthanks!!!#2020-07-0819:22fabraohello all, what is the magic about starting so fast in comparing with lein and boot ?#2020-07-0819:25souenzzo- less deps, clean design
- do less things (lein/boot do A LOT more stuff)
- cache classpath "in bash"
...#2020-07-0819:25borkdudefire up only one java process (when classpath is cached)#2020-07-0819:27souenzzodesign choise example:
lein help will build a classpath, start a JVM, load all plugins, ask to every plugin if it implements "help" and eventually show the help message
clj --help is handled without start any JVM#2020-07-0819:51fabraothis saved me time and memory, many thanks who made it#2020-07-0911:16ejelome...
Hello, I have a deps.edn question, I tried to replicate what I use in the terminal as config options with deps.edn, but it seems that the generated .js file logs some errors,
Here's the CLI:
clj -m cljs.main -o public/js/main.js -O advanced -c hello-world.core
Here's what I expected as equivalent deps.edn version:
{:deps {org.clojure/clojurescript {:mvn/version "1.10.764"}
reagent {:mvn/version "1.0.0-alpha2"}}
:main "hello-world.core"
:optimizations :advanced
:output-dir "public/js/out"
:output-to "public/js/main.js"}
Then I run it with:
clj -m cljs.main -co deps.edn
The generated .js file from CLI works fine, but the generated version from deps.edn options gives an error:
main.js:314 Uncaught TypeError: Cannot read property 'mg' of undefined
at Mb (main.js:314)
at Tn (main.js:699)
at Rn (main.js:698)
at Nn.h.Me (main.js:718)
at Function.Wn.A (main.js:710)
at Nn.h.Le (main.js:718)
at Function.Wn.j (main.js:710)
at ep (main.js:761)
at main.js:761
... and it doesn't seem to exit from the terminal (the terminal becomes unusable)#2020-07-0911:26euccastroyou may want to try asking in #clojurescript?#2020-07-0911:27ejelomehey, thanks, that's probably the right place#2020-07-0919:13psetmajAre there any default aliases or good ways to get some extra resource paths added to the classpath for all local repls started with the clojure command? (some key in ~/.clojure/deps.edn perhaps?)
I’d prefer not to need to add a -A flag to all invocations across the several IDEs I use#2020-07-0919:14alexmillerif you mean an alias that is automatically supplied, no#2020-07-0919:14alexmiller:paths in ~/.clojure/deps.edn will be used everywhere#2020-07-0919:15psetmajI thought :paths got merged across the candidate deps.edns with a “last one wins” strategy#2020-07-0919:16alexmilleroh yes, sorry#2020-07-0919:21psetmajDoes :extra-paths only exist for aliases? My use case is that I’ve got a local directory for various properties files that’s global to all projects (shared DB/service locations, and whatnot for local development, shared across several languages).#2020-07-0919:33alexmilleryes, only for aliases#2020-07-0919:48psetmajOkay, thanks for your responsiveness and time 🙂#2020-07-0922:12souenzzo@alexmiller I see that :git/url "file:///foo" isn't supported by gitlibs
I already find that it's a issue with clojure.tools.gitlibs.impl/clean-url
A PR/Patch is welcome about this problem?#2020-07-0922:13alexmillerI'm unclear why this is better than using :local/root#2020-07-0922:14alexmillergit deps let you get the dep from a remote git repo. if you already have it locally, just use it?#2020-07-0922:14alexmillerI guess a git dep would let you use multiple shas#2020-07-0922:14alexmillercan you walk back and say how you're using it?#2020-07-0922:15alexmillerstepping away for evening but will check later#2020-07-0922:19souenzzoLet's go:
I'm trying to deploy my application with clojure -Sdeps "{:deps {my/app {:git/url \"...\" :sha \"...\"}}}" -m my.app.start
But I'm on AWS/CodeCommit
CodeCommit on ephemeral EC2 recommends use https with credential provider
jgit do not support credential provider
so I'm trying to use git-cli to clone, then use local file
I can run directly but I want to keep my SHA control in the "start command line"#2020-07-0923:29souenzzoTL; DR: How to use git deps with CodeCommit?#2020-07-1011:27jeroenvandijk@U2J4FRT2T You have to put some magic in your .gitconfig . Something like:
[credential ""]
helper = !aws --profile dev1 codecommit credential-helper
And maybe enfore https (not sure, don’t remember):
[url ""]
insteadOf =
#2020-07-1011:28jeroenvandijkIf I remember correctly this did work with tools.deps#2020-07-1016:13kennyDoes anyone know of an up-to-date guide on deploying a jar to a private maven s3 repo? I've tried several different guides and they all reference deprecated or archived maven build plugins. It seems like such a simple task but there seems to be dozens of different ways to get it done.#2020-07-1016:31kennyOh interesting... it appears all I need to do is
aws s3api put-object --bucket my-bucket --key release/my-group-id/my-artifact-id/my-version/my.jar
#2020-07-1018:06kennyActually, tools-deps does not seem completely satisfied with the above approach, printing Download corrupted: Checksum validation failed, no checksums available#2020-07-1018:06kennyGuessing it expects some additional files in s3 that are not placed via the simple put-object action.#2020-07-1019:18drewverleeIf i have a lib calling a method that doesn't exist and i suspect its because two version of the lib exist and the one without it won, how can i verify this?#2020-07-1019:19drewverleethe stacktrace lists the class package and class name and i compared this against the output of Stree and their isn't a direct overlap#2020-07-1019:20drewverleelikley because stree is output a higher level of information#2020-07-1019:24seancorfield@drewverlee Which lib? Which class? Which method?#2020-07-1019:27drewverleemethod: ""com.google.common.collect.ImmutableList.toImmutableList()Ljava/util/stream/Collector;"
lib that calls it: pack/pack.alpha https://github.com/juxt/pack.alpha.git @ 0c20aa5#2020-07-1019:28drewverleewell more accurately:
[{:type java.lang.NoSuchMethodError,
:message
"com.google.common.collect.ImmutableList.toImmutableList()Ljava/util/stream/Collector;",
:at
[com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep
makeList
"BuildAndCacheApplicationLayerStep.java"
67]}]
#2020-07-1019:29drewverleeso google.cloud.tools.jib.builder calls com.google.common.collect.ImmutableList.toImmutableList and it doesn't exist. I assume that error isn't saying the arguments were bad in someway, its saying that method isn't there at all.#2020-07-1019:38seancorfieldAnd what Google libs show up in clojure -Stree for you?#2020-07-1019:38seancorfieldYou may need to run clojure -Strace and see what other versions of the various Google libs were considered.#2020-07-1019:40drewverlee> And what Google libs show up in clojure -Stree for you?
I see guava listed under one dep i just added to my project. Is that because Stree only shows the winner of the dep merge? I suppose i'm looking for everyone that tried to add it.
I'll look at Strace more carefully.#2020-07-1019:45drewverleei see Strace contains anyone that wanted to add the dep. isee it all over the place here.#2020-07-1019:50seancorfieldThe output of -Strace should show you every version of every library that it considered and why it made the decision it did to pick a particular version.#2020-07-1019:51seancorfieldhttps://guava.dev/releases/21.0/api/docs/com/google/common/collect/ImmutableList.html#toImmutableList-- shows that method was added in Guava 21.0#2020-07-1019:52seancorfieldSo you could just add a top-level dep on Guava to force a recent version, but I'm surprised tools.deps would pick a lower version, since I would expect the Google Cloud libs you're using to pull in at least that version or higher...#2020-07-1019:54seancorfieldAccording to Guava's docs, it's up to 28.0 now. 👀#2020-07-1019:54seancorfieldHmm, odd, we're using 29.0 at work...#2020-07-1019:55seancorfield21.0 is about 3 1/2 years old at this point. 29.0 came out in April this year @drewverlee#2020-07-1019:56seancorfieldWe've had to pin it as a top-level dependency because other things were bringing in older versions. Java's equivalent of "DLL hell" 🙂#2020-07-1101:05tolitiuswhat is a tools.depsy way of starting (--interactive) REPL-y connected to nrepl?
:repl {:extra-deps {nrepl {:mvn/version "0.7.0"}
reply {:mvn/version "0.4.4"}}
:main-opts ["-e" "(require,'[reply.main]),(defn,reply,[_,port,_],(reply.main/launch-nrepl,{:attach,(str,\"127.0.0.1:\",port)})),(ns,user)"
"-m" "nrepl.cmdline" "-i" "-f" "user/reply"]}}
this works (ish), but feels complex + does not exit nrepl once REPL-y is done (ctrl c / ctrl d)#2020-07-1110:57euccastrodo you want to start your nREPL server with the same command?#2020-07-1110:57euccastroI have this:
{:extra-deps {reply {:mvn/version "0.4.4"}}
:main-opts ["-m" "reply.main" "--attach" "1337"]}#2020-07-1110:58euccastrobut that's to attach to an already running nREPL#2020-07-1114:39tolitiusright nrepl + REPL-y connected to it. I do most of the work in a vi + terminal REPL#2020-07-1114:39tolitius*right => "start your nREPL server with the same command"#2020-07-1114:41tolitiusnrepl.cmdline optionally takes a repl-fn / -f for the interactive repling#2020-07-1114:56seancorfieldI think, for simplicity's sake, I'd be inclined to use two separate aliases: one to start an nREPL server (and leave it running in a window, or run it in the background), and a second alias to start REPL-y connected to that background process.#2020-07-1114:58seancorfieldI can understand wanting a single command, but I'd probably write a small shell script that just ran two clojure commands if it proved too annoying to run those two commands manually.#2020-07-1117:18tolitius@seancorfield definitely a possibility, but if I work with several (different) apps that's a twofold repl things to run and exit from
I am just looking for "lein repl" / "boot repl" repl-y like experience with deps.edn#2020-07-1117:21seancorfieldHmm, I never type into a REPL so I only care what my editor integration does. Why do you need REPL-y, specifically @tolitius?#2020-07-1117:21tolitiusI always type into repl )#2020-07-1117:22tolitiusit's connected to vi where I "finalize" my thoughts after I am done thinking in repl#2020-07-1117:23tolitiusand vice versa compile/reload from vi back to repl tp keep thinking#2020-07-1117:23seancorfieldI "think" in a "Rich Comment Form" in my editor -- like Stu Halloway talks about in some of his presos#2020-07-1117:30tolitiusyea, a lot of Clojure devs use an integrated playground to "think" that is just connected to nrepl that runs in the background. I like terminals, terminal repl works best for me since I am focused on small fragments of code at a time.
with REPL-y this experience is simple, so I am just looking to get there with deps.edn#2020-07-1117:30tolitiuswhich I did get with an above alias but it rather feels off + does not exit the nrepl on ctrl c/d#2020-07-1117:31seancorfieldI think you're stuck with the complex/almost-works invocation for now. I'll have a think about it on and off and come back if I figure out anything better.#2020-07-1117:31tolitiusyea, nothing urgent, I was just wondering whether I am missing something obvious, i.e. a flag, an option, etc..#2020-07-1117:31seancorfield(I don't use nREPL at all so it'll be a fun experiment/diversion)#2020-07-1117:31tolitiusgood luck and thank you )#2020-07-1119:28richiardiandrea@tolitius Hey 😉
I am using https://github.com/bhauman/rebel-readline
For your exact use case and I found some problems with multi-line expressions sometimes, I am pretty happy with it.
I have two aliases I can share (the second is for cljs) not in front of the keyboard now though. Will sent them soon.#2020-07-1216:33tolitius@richiardiandrea niice:
:repl {:extra-deps {nrepl {:mvn/version "0.7.0"}
cider/cider-nrepl {:mvn/version "0.22.4"}
com.bhauman/rebel-readline {:mvn/version "0.1.4"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]" "-i" "-f" "rebel-readline.main/-main"]}
this works: connects to an editor, reloads recompilations in repl from editor, and also I can now even see some color as a positive side effect )
any differences you found between rebel-readline and REPL-y (that REPL-y does better)?#2020-07-1217:27richiardiandreaNot much, as I mentioned sometimes rebel's multi-line support has glitches but it was never bothering so much to even open an issue (I should).
Nice thanks for sharing I do something similar myself.#2020-07-1217:34tolitiusthat's great, I am still mostly on boot due to little things like this in "deps.edn". but I am slowly moving over )
thanks a lot for a "rebel-readline" idea!#2020-07-1223:58richiardiandreamy pleasure 😄#2020-07-1316:55Space GuyWhen I used rebel-readline with a background process (Figwheel {:mode :serve} , http-kit, etc), the default rebel-readline/-main won't automatically end those processes if you quit with 'ctrl+d'.
Looks like REPL-y does (shutdown-agents) when done, I haven't checked what nrepl.cmdline does with that specific case
It's not a bug, just a difference (https://github.com/bhauman/rebel-readline/issues/141)
In my case I decided to start both then just exit when done:
(figwheel.main.api/start {:mode :serve} "dev")
(rebel-readline.main/-main)
(System/exit 0)
I currently run this in a clj dev.clj, otherwise a namespace with a main method is the same#2020-07-1321:35Ian Fernandezhello folks, some help on depstar pls (edited)
clj -C:build -A:build -m hf.depstar.uberjar target/abc.jar -C -m abc.core (edited)
this is giving me
Could not locate abc/core__init.class, abc/core.clj or abc/core.cljc on classpath.
but I have a src/abc/core.clj with (:gen-class)
some clue?#2020-07-1321:48hiredmanwhen you say it is giving you that, do you mean when you get that when building the uberjar or trying to run it?#2020-07-1321:57seancorfieldSince you're specifying -A:build you don't need to also specify -C:build -- -A means "all" so it already includes the effect of the -C option.#2020-07-1321:58seancorfieldWhat is your :build alias in your deps.edn @d.ian.b?#2020-07-1321:59Ian Fernandez:build {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}}#2020-07-1322:00seancorfieldAnd what exact output are you getting from that clj -A:build command? (I'm assuming the Could not locate... error is from trying to run the JAR?)#2020-07-1322:01Ian FernandezCompiling abc.core ...
Execution error (FileNotFoundException) at hf.depstar.uberjar/run$fn (uberjar.clj:331).
Could not locate abc/core__init.class, abc/core.clj or abc/core.cljc on classpath.
Full report at:
/tmp/clojure-5041631337400948497.edn
#2020-07-1322:02Ian Fernandezclj -A:build -m hf.depstar.uberjar target/abc.jar -C -m abc.core#2020-07-1322:04seancorfieldSo that says it's compiling norach.core -- which means depstar thinks that is the main class/namespace you told it to compile.#2020-07-1322:05Ian Fernandezops#2020-07-1322:05Ian Fernandezsrr#2020-07-1322:07Ian Fernandezops#2020-07-1322:07Ian Fernandezit was abc.core#2020-07-1322:07Ian FernandezI've tried to anonimize =(#2020-07-1322:07Ian Fernandezthe names#2020-07-1322:07Ian Fernandezand did it wrong#2020-07-1322:08seancorfieldIt really helps if you paste actual commands and actual error messages so you avoid transcription problems.#2020-07-1322:09seancorfieldI just did this:
$ clj -A:new app norach.core
... creates norach.core project ...
$ cd norach.core
$ clj -Sdeps '{:deps {seancorfield/depstar {:mvn/version "RELEASE"}}}' -m hf.depstar.uberjar target/abc.jar -C -m norach.core
Compiling norach.core ...
Building uber jar: target/abc.jar
Processing pom.xml for {norach/norach.core {:mvn/version "0.1.0-SNAPSHOT"}}
$ java -jar target/abc.jar
Hello, World!#2020-07-1322:09Ian Fernandezclj -A:build -m hf.depstar.uberjar target/norach.jar -C -m norach.core
Compiling norach.core ...
Execution error (FileNotFoundException) at hf.depstar.uberjar/run$fn (uberjar.clj:331).
Could not locate norach/core__init.class, norach/core.clj or norach/core.cljc on classpath.
Full report at:
/tmp/clojure-5874121970680475144.edn
#2020-07-1322:10seancorfieldSo that indicates that your deps.edn file is not setup correctly -- so norach/core.clj is not on your classpath.#2020-07-1322:10seancorfieldCan you run this: clj -m norach.core#2020-07-1322:12Ian FernandezExecution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate norach/core__init.class, norach/core.clj or norach/core.cljc on classpath.
Full report at:
/tmp/clojure-13768805695698659229.edn#2020-07-1322:12Ian Fernandez=(#2020-07-1322:12Ian Fernandeznope#2020-07-1322:12seancorfieldThat confirms it's not a depstar problem -- it's a problem with your project.#2020-07-1322:12Ian Fernandezthanks#2020-07-1322:13seancorfieldYou're sure you have src/norach/core.clj and that file starts with (ns norach.core ...) ?#2020-07-1322:13Ian Fernandezdammit, misstypo#2020-07-1322:13Ian Fernandezns norahc.core#2020-07-1322:13Ian Fernandezdammit#2020-07-1322:13Ian Fernandezthanksss#2020-07-1322:18Ian FernandezThanks @seancorfield 😃#2020-07-1411:54eval2020Any chance that tda will support providing multiple config files? e.g. by allowing CLJ_CONFIG to be ~/.clojure:/considered/first
This would be great to share a deps.edn via dotfiles or in a monorepos.
context: I'd like to curate a deps.edn for the team I'm working with. This file would contain aliases and overrides that are now duplicated across repositories.
This file should exist along a user's ~/.clojure/deps.edn .
E.g. CLJ_CONFIG=~/.clojure:~/.clojure/work clojure -Sdescribe would result in ...:config-files ["<global>/deps.edn" "~/.clojure/deps.edn" "~/.clojure/work/deps.edn" ?"$PWD/deps.edn"] ensuring that within the team we share aliases while allowing for user-specific aliases and versions (e.g. for editor setup).
In a monorepos I can see this being helpful as well. E.g. with the following project setup and a global export CLJ_CONFIG=~/.clojure :
.
├── shared/deps.edn
└── subproject1/deps.edn
doing export CLJ_CONFIG=$CLJ_CONFIG:/path/to/project/shared as part of a project setup (or in scripts) would allow one to move through the project and have a config-files collection of ["<global>/deps.edn" "~/.clojure/deps.edn" "/path/to/project/shared/deps.edn" ?"$PWD/deps.edn"]#2020-07-1412:15dominicmWrite a tool to update the deps edn file in each repo#2020-07-1412:43eval2020@U09LZR36F thought about going that way but seems hard to maintain across a lot of repositories...#2020-07-1413:43dominicmWhy's that#2020-07-1514:05eval2020In our case it would be the amount of repositories (50-ish). Also the contents of the deps.edn: some contain a linter alias, others a clj-kondo, any docs for existing alias. And then doing it for every update to the shared stuff.
Have you successfully applied this approach?#2020-07-1515:23dominicmNot to 50 or so repos. But if you want people to actually update, then you probably want to push it into their repo, instead of hoping that they will update it on their local machine.#2020-07-1412:25alexmillerNot planning to make CLJ_CONFIG into a path but we may at some point add some way to have additional deps files#2020-07-1412:40eval2020good to hear it's being considered, thanks#2020-07-1415:33seancorfieldI would stop relying on CLJ_CONFIG if I could specify one or more additional deps.edn files that went into this slot: <system> <user> <additional file(s) here> <project>.#2020-07-1416:37alexmillerI prefer that way of thinking about it to path idea above#2020-07-1416:38alexmillerwe had lots of questions the last time we talked through this. Some of those have been resolved since, some have not.#2020-07-1416:38alexmillerwe = Rich and I, that is#2020-07-1419:42eval2020The scenarios I sketched could then be solved by specifying an alias or wrapper like clj --deps project-deps.edn ?#2020-07-1420:06alexmillerwell that is the question - how would you want to work with this in practice#2020-07-1502:16salamis it possible to automatically install clojure cli on jenkins as a custom tool where the jenkins user doesn't have enough privileges (i.e., neither root nor on the sudo list) to do so?#2020-07-1502:18salamthis is what we are getting right now:
[clojure_cli] $ sh -e /var/lib/jenkins/tools/com.cloudbees.jenkins.plugins.customtools.CustomTool/clojure_cli/hudson7631100789357617495.sh
[...]
You need to be root to perform this command.
[...]
Installing libs into /usr/local/lib/clojure
install: cannot create directory /usr/local/lib/clojure: Permission denied
#2020-07-1502:28seancorfield@abdusalam I believe you can run the script with the --prefix option and a directory to which you have write access.#2020-07-1502:32seancorfield(yes, I just tested this works)#2020-07-1502:32salamah, indeed! (https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/linux-install.sh)
thanks, @seancorfield#2020-07-1619:51markbastianIf I want to publish a tools deps project as a jar to a repo (e.g. clojars) and I have an up-to-date pom.xml, what is the "right" artifact to publish if you are using git coordinates for your dependencies? Do I need to create an uberjar and publish that? Is there a way to package the git deps src in a regular jar target? This is assuming a library project with git dependencies.#2020-07-1620:01alexmillerthere is no way to do that#2020-07-1620:01alexmillergit deps are only usable from source deps#2020-07-1620:02alexmilleras soon as you say "artifact", you're talking about maven and you need to depend only on other artifacts#2020-07-1620:03markbastianMakes sense. Seems like an uberjar as lib or jar + git deps as lib would be a mess.#2020-07-1620:17kennyIs there a Maven command or Clojure lib someone made that uploads a jar file to a S3 bucket? All the tools I've found seem like they expect to build the jar for you.#2020-07-1620:18alexmillerwell you can just do an aws s3 cp if that's sufficient#2020-07-1620:20kennyI tried that and receive this message from tools-deps "Download corrupted: Checksum validation failed, no checksums available." It actually all works (i.e., jvm starts up with the jar on the cp) but it'd be nice to do it the correct way.#2020-07-1620:20kennyOr maybe that's a bug in tools-deps?#2020-07-1620:24alexmillerwell if you just upload the jar without the checksum files, you will see this message#2020-07-1620:26kennyI figured. Is there something out there that can do this the correct way to avoid this warning?#2020-07-1620:28alexmillerthe correct way is to compute an md5 or sha1 checksum for the jar and upload those (and ideally to gpg sign all of those files as well). I do not know of a tool that does that (although I have started working on one)#2020-07-1620:29alexmilleran example of what goes into maven central is something like this: https://repo1.maven.org/maven2/org/clojure/tools.deps.alpha/0.8.677/#2020-07-1620:29alexmillerhow that all gets created and uploaded is quite complicated, and clojars does much less (and even less is really required)#2020-07-1620:30alexmillerreally, just the jar and the md5 or sha1 file (and ideally the pom) are probably sufficient#2020-07-1620:31alexmilleryou can use a tool like sha1sum or md5sum to make that#2020-07-1620:32alexmillerthere is also a version metadata file that should be updated too (like https://repo1.maven.org/maven2/org/clojure/tools.deps.alpha/maven-metadata.xml) if you want something like RELEASE version to work.#2020-07-1620:34kennyWowza. Did not realize what I was getting myself in to. Thank you for all this info. I assume the work you started on is not public yet?#2020-07-1620:35alexmillerno, nothing usable#2020-07-1620:37alexmillerI would like to handle a) clojars b) s3 and c) maven central in that order which I think handles both the most common and easiest items first#2020-07-1620:37alexmillerthere are multiple options on each and it is way harder than it should be for sure#2020-07-1620:42kennyWill try the jar + hash suggestion. And try to restrain myself from building something complete. Thanks again 🙂#2020-07-1621:12dominicmMaybe I should build a tool on top of pack. It would force me to finish the api.#2020-07-1622:11drewverleeFor deps that need to be kept in sync, or even in times where its likely two things should udpate together, what strategy have people used?
My first thought would be that deps wouldn't be responsible for this and this more when you need a tool like lein. But if i only have this issue maybe soemthing a tad more lightweight.#2020-07-1622:31seancorfield@drewverlee I put them next to each other in deps.edn with a comment explaining the relationship.#2020-07-1622:32seancorfield(it's a project/application concern, not a tooling concern really -- so it's more about making sure people know how/when to change stuff)#2020-07-1622:34seancorfieldBTW, that's also why I'm generally against tools that update my configuration files automatically -- those tools cannot know about such constraints, e.g., I may have very valid reasons for keeping two particular dependencies on older versions together, even if newer versions of one or both exist.#2020-07-1623:10drewverleeSeems reasonable, I guess I should start with the comment and add more as needed. Thanks @seancorfield#2020-07-1708:51danieroux@kenny This is how I solved it:
mvn org.apache.maven.plugins:maven-install-plugin:3.0.0-M1:install-file -Dfile=dev-local-0.9.172.jar -DlocalRepositoryPath=s3_m2_repository
aws s3 sync s3_m2_repository
#2020-07-1715:29kennyOh, nice! This seems like it'd upload all the necessary files. Sweet.#2020-07-2102:30kennyThis approach still results in the Download corrupted: Checksum validation failed, no checksums available warning message though 😞#2020-07-1709:06danieroux@drewverlee I have entries like these:
org.slf4j/slf4j-api {:mvn/version "1.7.14" :required-by :datomic-cloud}#2020-07-1721:03alexmillerhey clj super fans, there is a new version of clj (and tools.deps) out and I'd love to have some people try it and verify no dep tree differences. you can install the latest dev release with:
brew uninstall clojure
brew install --devel clojure/tools/clojure
#2020-07-1721:03alexmillerhey clj super fans, there is a new version of clj (and tools.deps) out and I'd love to have some people try it and verify no dep tree differences. you can install the latest dev release with:
brew uninstall clojure
brew install --devel clojure/tools/clojure
#2020-07-1721:24seancorfield@U064X3EF3 I expect you know about this
Warning: Calling 'devel' blocks in formulae is deprecated! Use 'head' blocks or @-versioned formulae instead.
Please report this issue to the clojure/tools tap (not Homebrew/brew or Homebrew/core), or even better, submit a PR to fix it:
/usr/local/Homebrew/Library/Taps/clojure/homebrew-tools/Formula/clojure.rb:7
#2020-07-1721:26alexmilleryeah#2020-07-1721:26alexmillerbrew breaks things faster than I can fix them#2020-07-1721:27seancorfieldThen I got
==> Downloading
######################################################################## 100.0%
Error: clojure 1.10.1.536 is already installed
To upgrade to 1.10.1.561, run `brew upgrade clojure/tools/clojure`.
==> `brew cleanup` has not been run in 30 days, running now...
and it did not actually upgrade clojure 😐#2020-07-1721:28seancorfield(! 1406)-> clojure -Sdescribe
{:version "1.10.1.547"
:config-files ["/usr/local/Cellar/clojure/1.10.1.547/deps.edn" "/Users/sean/.clojure/deps.edn" ]#2020-07-1721:28seancorfieldWhich is not 561#2020-07-1721:29alexmilleroh, sorry#2020-07-1721:29alexmillerbrew upgrade --devel clojure/tools/clojure#2020-07-1721:29seancorfieldNope, that doesn't work either.#2020-07-1721:29seancorfieldError: invalid option: --devel
#2020-07-1721:29alexmillerwell then brew uninstall clojure first#2020-07-1721:30alexmillerthen install --devel#2020-07-1721:30seancorfield(! 1413)-> clojure -Sdescribe
{:version "1.10.1.561"
:config-files ["/usr/local/Cellar/clojure/1.10.1.561/deps.edn" "/Users/sean/.clojure/deps.edn" ]
Had to uninstall then install --devel#2020-07-1721:31alexmilleryeah, that's the correct one#2020-07-1721:06alexmillerthe change of note here is I have rewritten how exclusions are handled if the same dep shows up more than once in the dep tree with different exclusion sets. this is uncommon, but possible and most commonly happens by having the dep with exclusions in one place and with no exclusions in another. in the past, what happened was somewhat arbitrary - depending which you first encountered you may get the dep or not.#2020-07-1721:08alexmillerthe updated behavior is that you will get the intersection of exclusion sets from all occurrences, so in the case above you will always get the dep#2020-07-1721:09alexmillerin the process of implementing this, I've also substantially refactored the deps expansion code to make it easier to work on and comprehend. while those changes should be a no op, I did change a lot of code and well, you know.#2020-07-1721:10alexmillerso if anyone has time to do a sanity check, that would be cool#2020-07-1721:12alexmilleralso, in the process of checking these changes, it's become clearer to me that -Stree is likely to lie to you about what happened in exclusion cases like the above, because it's based on the lib map, which is an inverted tree that loses exclusion path info. I'm going to rewrite it in terms of the trace data and then it will be both correct but can also include substantially more useful information (akin to tools.deps.graph trace mode). but going to do other work first.#2020-07-1721:32seancorfieldA quick test shows that it doesn't attempt to download any new dependencies for our monorepo so that looks sane.#2020-07-1721:33alexmiller:)#2020-07-1721:34alexmillerhaving a new clj should make everything stale (so effectively -Sforce), but maybe -Sforce too?#2020-07-1721:35seancorfieldI'm running our script that checks for updated versions of deps -- it builds new projects with our deps and then runs -Stree in those so there no cache. I'll let you know if that turns up any differences.#2020-07-1721:36seancorfieldLooks good. It detected a new version of org.apache.axis/axis and all the other dependencies remained unchanged.#2020-07-1721:36seancorfield(that script hammers clojure -Stree 🙂 )#2020-07-1721:43seancorfieldJust for good measure, I blew away the cache folder in every subproject in our monorepo and rebuilt things. No changes in deps as far as I can tell.#2020-07-1721:45alexmillerThanks, good to have someone else try it :)#2020-07-1721:45seancorfieldYou know me -- always happy to be on the bleeding edge!#2020-07-1722:06kennyI can try this. @seancorfield how did you check for differences? Set diff of classpath?#2020-07-1722:16seancorfieldJust compared -Stree before and after.#2020-07-1722:17seancorfieldBut also ran our test suite to see if any new dependencies got downloaded (none).#2020-07-1802:24kennyNo new deps were printed when running clojure -Spath -Sforce . Also compared the classpath of the old and the new and they are identical.#2020-07-1802:47alexmillerThanks, as expected,#2020-07-2102:31kennyAny idea why clojure -Spom would sometimes result in duplicate entries? e.g., something like this:
...
<build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
</build>
...#2020-07-2102:32alexmillerare you making a new one or syncing?#2020-07-2102:33alexmillerand which clj version?#2020-07-2102:33kennyThis resulted from this:
rm -rf pom.xml
clojure -Spom
clojure -Spom
#2020-07-2102:33alexmillerthe answer to your question is certainly that it's a bug :)#2020-07-2102:33kenny1.10.1.561#2020-07-2102:34kennyThis is my deps.edn
{:paths ["src"]
:aliases {:jar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}
:main-opts ["-m" "hf.depstar.jar" "s3-mvn-upload.jar"]}}}#2020-07-2102:34alexmillerthere was a change to this in the prior version, which I actually have found was bad and a pretty subtle bug. I have a fix for it but haven't gotten it into master yet.#2020-07-2102:35alexmilleris it ok after the first clojure -Spom ?#2020-07-2102:35kennyNo. It creates this:
<build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
</build>#2020-07-2102:35kennySecond run:
<build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
</build>#2020-07-2102:36alexmilleryeah, I know why it's doing this#2020-07-2102:36alexmillerwell I will try to get a fix out for it soon#2020-07-2102:36kennyAny idea which version doesn’t have this? 2 versions back?#2020-07-2102:37alexmillercheck the changelog#2020-07-2102:38alexmillerhttps://github.com/clojure/tools.deps.alpha/blob/master/CHANGELOG.md#2020-07-2102:39alexmillerprior was tools.deps 0.8.677 which corresponds to 1.10.1.510#2020-07-2102:39alexmilleruse https://github.com/clojure/homebrew-tools#version-archive-tool-releases to fall back on mac#2020-07-2102:39kennyYeah was looking for the mapping. Is there a place for tdeps -> brew version?#2020-07-2102:39alexmillerat the top of that changelog is a link to the other changelog#2020-07-2102:40alexmillerand vice versa#2020-07-2102:40kennyPerfect. Thank you!#2020-07-2102:41alexmillerthere was a bug in initial gen in the prior version too though so you may not be any happier#2020-07-2102:42kennyHaha alright. At some point it worked, at least for my simple use case, since I’ve been using -Spom to deploy libs to clojars. Or perhaps I’ve just never noticed it generating something incorrect 😬#2020-07-2102:52seancorfieldLuckily http://clojars.org doesn't care I guess 🙂#2020-07-2102:52kennyFor those wanting to simply upload a jar file to a S3 Maven repo, I created a small Clojure lib to do (some of) the right things. You can paste it into a bb script if you want it to go a bit faster. https://github.com/kennyjwilli/s3-mvn-upload#2020-07-2102:54kennyFolks using Datomic’s dev-local may find that useful ^#2020-07-2216:50practicalliI'm using https://github.com/seancorfield/clj-new to add a new file/namespace called handlers to an existing project, practicalli/banking. I assume this is the right command given an alias called :new in my user level deps.edn file. It seems to work okay.
clojure -R:new -m clj-new.generate ns=practicalli.handlers
cc @seancorfield#2020-07-2216:54seancorfieldThat's a reasonable approach. I'd probably add a :gen alias so I didn't have to type -m clj-new.generate each time.#2020-07-2216:54seancorfield(caution: the generators are very primitive and not particularly well thought out -- I don't actually find them valuable and I would prefer they not be promoted as a way to do anything!)#2020-07-2216:55seancorfieldAt some point, they'll probably be completely rewritten (in a new ns, naturally).#2020-07-2217:21practicalliOk, thanks. I only just used it today for the first time successfully. I tried previously but was getting an error as I didnt include any kind of clj-new alias, so content in just getting it working for myself 🙂#2020-07-2217:22practicalliI am more interested in writing some templates to use with clj-new, so will focus on that instead, which I assume is quite stable.#2020-07-2217:31seancorfield@jr0cket Yup, if you run into any problems, feel free to DM me about it.#2020-07-2217:31seancorfieldThe built-in app/lib templates are probably a good place to get started. Also let me know if you have suggestions for improving the README around writing templates.#2020-07-2221:21markbastianWhat's the best way to run a socket or nrepl server locally? The scenario is that we have a windows user with a deps.edn project that is having trouble getting a repl started in IntelliJ. I was thinking he could just run a socket REPL and jack in. I see aliases for nrepl https://nrepl.org/nrepl/usage/server.html as well as socket repl directions https://clojure.org/reference/repl_and_main#_launching_a_socket_server. Any recommendations besides having an accident and getting a Mac instead?#2020-07-2221:43markbastianShould this be the right invocation? clj -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"#2020-07-2221:48markbastianGot it! From https://oli.me.uk/exploring-repl-tooling-with-prepl/:
clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/repl}"#2020-07-2303:06salama better approach would be to create an alias in the project deps.edn: https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L66
i'm a cursive user myself and this is how i start a socket repl server in the terminal ( clojure -O:socket ) and connect cursive (a socket repl client) to it by creating a remote socket repl run configuration in intellij idea.#2020-07-2315:26markbastianNice. Thanks! I think 90% of the answers are in Sean Corfield's deps.edn file.#2020-07-2315:27salamyup, i personally found a lot of gems in there. 🙂#2020-07-2305:48tengstrandI can set up my deps.edn at the root so that it includes a "module" (that has its own deps-edn) like this:
:aliases {:dev {:extra-deps {env/dev {:local/root "environments/dev"}}}
:test {:main-opts ["-m" "polylith.clj.core.cli.poly" "test"]}}
Then I can run:
clj -A:test:dev
...and it will run my tests (that is what the 'test' alias does for me).
But the deps.edn under environments/dev contains its own aliases with some :extra-paths that I want to include also. Is that possible? I imagine that I need to do something like this (which doesn't work):
clj -A:test:dev:dev/local-alias
...where local-alias is defined as an alias in environments/dev/deps.edn:
:aliases {:local-alias {:extra-paths ["../../components/help/src"]}
#2020-07-2314:34seancorfieldNo, aliases in dependencies are not relevant. That would be kind of dangerous as you'd suddenly become vulnerable to whatever aliases developers added to their own projects, which could interfere with your own user-level aliases. And you'd also be affected if a dependency switched from Leiningen to Clojure CLI or vice versa -- which runs completely counter to any notion of reproducibility.#2020-07-2406:04tengstrandOkay, but here I suggest to use existing aliases that lives in environments/dev/deps.edn, that we "import" from deps.edn at the root. Then we don't "mess up things from the outside" just selecting aliases within that "imported" deps.edn. If I "cd environents/dev" first, then I have access to all those aliases which have the purpose to add :extra-paths which I need. But here I have several "sub modules" (not just the one in this example) and wants to do the same from the root deps.edn without having to manually merge all those deps.edn files into one huge deps.edn at the root.#2020-07-2406:06seancorfieldAs I said @U1G0HH87L "That would be kind of dangerous".#2020-07-2406:08seancorfield:local/root isn't special -- it's just another way to identify a dependency. You're asking for dependencies that include deps.edn to have their aliases combined with the current project. That's not a practical way to deal with reproducibility in project builds.#2020-07-2406:11tengstrandThen I guess I need to solve it with code generation or create my own REPL or similar.#2020-07-2406:15seancorfieldWhat we do at work, with our 100k+ line monorepo containing 30+ subprojects, is that we have a special subproject (called versions) that contains the "master" deps.edn that has all the aliases needed by all projects, including :defaults which brings in all the :override-deps to pin versions etc, and then all the other subprojects really just have :deps and an alias for :test-deps to add anything needed for testing that subproject (`versions/deps.edn` has a :test alias for all the common testing infrastructure). Then we use CLJ_CONFIG=../versions clojure -...aliases in each subproject.#2020-07-2406:16seancorfieldCLJ_CONFIG lets you treat a specific folder as if it contains your user-level deps.edn (and so any actual ~/.clojure/deps.edn file is ignored -- which is good: the monorepo then control all dependencies and aliases!)#2020-07-2915:24tengstrandThanks for the information, can be very useful!#2020-07-2308:54p-himikThere's no way to prevent clj from reading deps.edn in the current directory, is there?#2020-07-2309:56borkdude@p-himik don't think so.#2020-07-2309:59p-himikThanks. That's seems a bit inconvenient because if I want to run some tool with -Sdeps (like Capsule in my particular case) within my project, I cannot really do it without mixing two different classpaths.#2020-07-2310:02plexusit's fairly annoying, I've had scripts that temporarily move deps.edn out of the way because of this...#2020-07-2310:04borkdudedeps.clj, an unofficial re-implementation of the clj script does have an option to choose a different deps.edn file than the one in the current directory#2020-07-2310:05vlaaadI think setting :paths in aliases to replace source path from deps.edn is as close as it gets to what you want#2020-07-2310:06vlaaadand :deps#2020-07-2310:06borkdudeit's also possible to remove deps from the classpath in an alias:
https://github.com/borkdude/babashka/issues/152#2020-07-2310:06vlaaadsee https://clojure.org/reference/deps_and_cli#_aliases (“-A” section)#2020-07-2310:07p-himikBut then I'd have to restructure my whole deps.edn just to make some thirdparty tool happy. That's not good.#2020-07-2310:08vlaaadnot sure I’m following, can you put that third-party tool as a user-level alias?#2020-07-2310:09p-himikThe tool itself - yes. The thing is, it must be run in the presence of deps.edn. But it would be best to not let clj (and I run the tool using clj) also interpret deps.edn.#2020-07-2310:10p-himikThe tool itself is used via -Sdeps, so it's not in the deps.edn.#2020-07-2310:08borkdudeyou can also manually construct the classpath (using a script) and pass it to -Scp#2020-07-2310:08vlaaadthat way you don’t touch your project’s deps.edn at all#2020-07-2310:14p-himikOh, it's now possible to use :deps in aliases. There's hope!#2020-07-2310:15borkdudeThat overrides the base deps?#2020-07-2310:15p-himikYep, seems like it.#2020-07-2310:16p-himikSo basically I should be able to mimic an empty deps.edn. Will experiment later today.#2020-07-2310:16borkdudenote that you will still have clojure and spec on the cp by default#2020-07-2310:17p-himikHmm, right. But that should've really interfere with tools that are already meant to be run with clj.#2020-07-2310:27borkdudeI also use deps.edn for babashka projects, the extra deps don't hurt, but they can make searching the classpath a tad slower (like a few ms maybe)#2020-07-2310:29p-himikIn my case, they don't anything really bad as well, but I really want to get rid of those slf4j warnings.#2020-07-2312:27alexmilleryes, using :deps / :paths in an alias is what we call "tooling" and will replace the project :deps/:paths (for the purpose of using a tool that shouldn't use the project classpath)#2020-07-2312:28alexmilleruntil yesterday, you could only use those via -A, but as of the latest dev release yesterday, tooling -T is now a thing#2020-07-2312:38p-himikAwesome, thanks!#2020-07-2312:40alexmillermore to come on that and other changes in latest, just been kind of busy with ... other things :)#2020-07-2313:06Tamizhvendan SHi, I am getting FileAlreadyExistsException while trying to build an uberjar
Here is my deps.edn file
{:paths ["src" "resources"]
:deps {io.appium/java-client {:mvn/version "7.3.0"}
org.clojure/data.xml {:mvn/version "0.0.8"}
metosin/reitit-ring {:mvn/version "0.5.1"}
metosin/reitit-middleware {:mvn/version "0.5.1"}
metosin/reitit-spec {:mvn/version "0.5.1"}
mount {:mvn/version "0.1.16"}
ring/ring-jetty-adapter {:mvn/version "1.7.1"}
org.slf4j/slf4j-simple {:mvn/version "1.7.30"}}
:aliases {:dev {:extra-paths ["dev"]
:extra-deps {org.clojure/tools.namespace {:mvn/version "1.0.0"}}}
:uberjar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}
:main-opts ["-m" "hf.depstar.uberjar"]}}}#2020-07-2313:07Tamizhvendan SHere is the command that I am running clj -A:uberjar my-app.jar#2020-07-2313:08Tamizhvendan SBuilding uber jar: my-app.jar
Execution error (FileAlreadyExistsException) at com.sun.nio.zipfs.ZipFileSystem/createDirectory (ZipFileSystem.java:422).
META-INF/versions/9/
#2020-07-2313:09Tamizhvendan SAm I missing anything obvious here?#2020-07-2317:19cap10morganWhat are folks using to store / set their project's version number so it makes it into the generated pom.xml (in an automated way; think CD pipeline, not hand-editing the pom.xml)?#2020-07-2317:31seancorfieldI'm treating the pom.xml as the "system of record" for the version number and deriving it elsewhere from the POM file.#2020-07-2317:33seancorfield(and that's the workflow that I use for all my deps.edn-based projects that get deployed to clojars -- note that Contrib libs such as core.cache, core.memoize, tools.cli that I maintain already follow this workflow with "Maven Release" process in CI editing the pom.xml to change the version number explicitly based on the version you enter in that Jenkins form)#2020-07-2319:53cap10morgan@seancorfield OK, thanks. What tools do you use to read / bump it in the POM file? Curious if you have anything resembling the lein release workflow going there.#2020-07-2322:35slimslenderslackshey @cap10morgan, we're treating git tags as the source of truth for versions. When we deploy, the version in the generated pom.xml is made to be in sync with "git describe". Since the pom.xml is only ever needed for the deploy action, the pom.xml doesn't actually have to be checked in.#2020-07-2322:42seancorfieldNote that for http://clojars.org and also for http://cljdoc.org there's an expectation that pom.xml contains a lot more than just the dependencies and the version, so having it as a "loose" file outside git would be... ill-advised if the generation process doesn't include all that information (which lein does put there, but clojure -Spom does not, so I'm talking about deps.edn projects -- note that clj -A:new would generate initial pom.xml files with all the appropriate fields for http://clojars.org and http://cljdoc.org)#2020-07-2322:51slimslenderslacksYa, our process is still using lein for the pom.xml generation. That's super interesting @seancorfield - I was probably going to hit this in the near term as I had been planning on migrating to use clojure -Spom. Thanks!#2020-07-2322:53seancorfieldclojure -Spom is fine for updating <dependencies> in an existing pom.xml file but it only generates a very bare-bones version if no file exists. That's why I added a full-featured pom.xml to the templates that clj-new uses for app, lib, and template.#2020-07-2322:57slimslenderslacksWhen git describe is the source of truth for version maybe we'd compose clojure -Spom to update the dependencies, with a git-describe->pom-version , and keep a clj-new generated pom.xml checked in.#2020-07-2323:01slimslenderslacksIt sounds like a checked in pom.xml might currently be the best place to version the extra metadata that clj-new adds.#2020-07-2323:05seancorfieldclj-new creates a baseline pom.xml. It never updates it after that.#2020-07-2323:06seancorfieldclojure -Spom updates an existing pom.xml with the latest dependencies (from deps.edn) so as long as you always do it prior to a build/deploy cycle, I guess those changes don't really matter.#2020-07-2323:07seancorfieldYour git describe update also seems to be "do it before build/deploy" but don't really care about the changes.#2020-07-2323:08seancorfieldHowever, a lot of people would be very confused by a pom.xml file in a repo that didn't match the dependencies in deps.edn and/or didn't seem to reflect the current released version 🙂#2020-07-2323:08seancorfieldSo whether you check it in or not after those changes probably depends on whether it's going into a public repo I suppose 🙂#2020-07-2323:12slimslenderslacksYa, I agree. It could be confusing to see a pom.xml with a weird version. You'd almost want the checked in copy to have a version of :project-uses-git-tags , and maybe have an empty dependencies, so it appears to be there just to define the extra maven stuff that you can't derive from elsewhere.#2020-07-2323:13slimslenderslacksI don't mind having a file with the correct version, I was more just trying to avoid the commits that go along with those workflows.#2020-07-2323:14slimslenderslacksOur unit tests and commit messages are the signals the produce Tags, and Tags are the signals that create new publishes to maven repos.#2020-07-2323:15slimslenderslacks(from a CI/CD perspective)#2020-07-2323:23seancorfieldYup. I think internal CI/CD processes are under different constraints and, unless you are publishing every successful build to (public) Maven Central / Clojars and also publishing every version to a (public) GitHub repo or similar, for internal releases you can definitely just keep a baseline pom.xml under version control and update that as needed for each build to your repo (and throw it away afterward).#2020-07-2320:00alexmillerthere are maven plugins for that although it's all pretty tedious#2020-07-2320:01alexmillerthe mvn release / deploy infrastructure does a lot for you but you have to buy pretty far into it all the way for it to be worth doing#2020-07-2320:01alexmiller(but it's a lot less fragile than the lein release stuff)#2020-07-2320:02cap10morganThanks Alex. lein release can indeed be fragile. 🙂#2020-07-2320:12alexmillerthe idea to put the version inside a file was probably bad from the beginning#2020-07-2320:13seancorfield@cap10morgan For my clojars-deployed stuff, I just update it as part of my general release process where I'm updating the README and the CHANGELOG and I update both the <version> and the <tag> fields in the pom.xml file all at the same time.#2020-07-2320:14seancorfield(so I grep for <old-version>|<new-version> so I can see all the places that refer to the current release and what will be the next one, and then I can visually verify that all references get changed)#2020-07-2320:15seancorfieldIn most of my libs, that includes several places in the README and usually in the documentation files as well that refer to installation/getting started and when new features were introduced.#2020-07-2320:19alexmillerI've automated that step on some libs :)#2020-07-2320:21seancorfieldI still have artisanal documentation 🙂#2020-07-2415:07drewverleeGiven a deps in project A that has a :local/root to another deps in project B. If i update a dep in project B and start my repl in poject A i would expect to have the new dep from project B be used. This doesn't seem to be the case. Any ideas what could cause this?#2020-07-2415:11alexmilleryes, you need to -Sforce to recompute the classpath#2020-07-2415:12alexmillerright now, changes in transitive local deps are not automatically discovered#2020-07-2415:12drewverleegotcha. thanks alex#2020-07-2415:15borkdudeI saw a mention of -T in this channel. Where is this documented?#2020-07-2415:23alexmillerin not yet published docs :)#2020-07-2415:24alexmilleronly exists in the newest dev release of clj#2020-07-2415:24alexmillerso I wouldn't tell anyone to use it yet#2020-07-2415:24alexmillerbut it's exactly what's supported now via -A for :deps and :paths in aliases, it just has an independent name now#2020-07-2415:58borkdudeI think I'll just wait for the docs 🙂#2020-07-2416:02alexmillerit's not really giving you anything new that you didn't already have :)#2020-07-2416:05alexmillerthere are other new and interesting things in this release though :)#2020-07-2416:06vlaaadNow I'll have to go through the changes...#2020-07-2416:07alexmillerwriteup is coming#2020-07-2616:37kennySomething strange is going on with s3 deps and tools-deps. I’m on 1.10.1.561. Given this deps.edn,
{:deps {com.datomic/dev-local {:mvn/version "0.9.184"}
com.datomic/client-cloud {:mvn/version "0.8.102"}}
:mvn/repos {"kwill-mvn" {:url ""}}}
the output of clojure -Stree -Sforce is
org.clojure/clojure 1.10.1
org.clojure/core.specs.alpha 0.2.44
org.clojure/spec.alpha 0.2.176
com.datomic/dev-local 0.9.184
com.datomic/client-cloud 0.8.102
com.cognitect/anomalies 0.1.12
com.datomic/client 0.8.111
...
It is not picking up the dev-local deps specified in the jar’s pom.xml. If I change the dev-local dep to be a local/root pointing to the Maven jar, I get the correct tree.
{:deps {com.datomic/dev-local {:local/root "/Users/kenny/.m2/repository/com/datomic/dev-local/0.9.184/dev-local-0.9.184.jar"}
com.datomic/client-cloud {:mvn/version "0.8.102"}}
:mvn/repos {"kwill-mvn" {:url ""}}}
Output of clojure -Stree -Sforce
org.clojure/clojure 1.10.1
org.clojure/core.specs.alpha 0.2.44
org.clojure/spec.alpha 0.2.176
com.datomic/dev-local /Users/kenny/.m2/repository/com/datomic/dev-local/0.9.184/dev-local-0.9.184.jar
org.clojure/tools.analyzer 0.6.9
com.google.errorprone/error_prone_annotations 2.3.4
...
Note how dev-local is now bringing in deps and previously it was not.#2020-07-2617:32hiredmanIf I recall, maven doesn't use the pom out of a jar, it keeps the pom alongside the jar, so it looks like you are deploying your jar to your maven repo with a different pom then you are building into it#2020-07-2617:44kennyAh, that does make sense. From a previous discussion with Alex, I was under the impression that I only needed to upload a jar & a hash to s3 to get it to work.#2020-07-2618:43hiredmanThat may be the minimal required for maven to download the jar, but more may be required to communicate dependencies#2020-07-2622:45alexmillerI think you may actually just be seeing bad output from -Stree#2020-07-2622:47alexmillerin general, -Stree will only list a dep once (even when it's included under multiple branches) - and it's pretty arbitrary where you will see it. for these two libs there are many shared deps.#2020-07-2622:49alexmillerthese two deps particularly share deps but with different exclusion sets, a case where this version of clj newly makes better, more intentional and reproducible choices. Unfortunately, -Stree is built from the lib map, which does not retain the original path structure and it's quite likely to see deps incorrectly under exclude points (because they are included at other places). -Stree is actively misleading in these cases. (the actual set of libs is right though!)#2020-07-2622:51alexmillerwhat I'm trying to say is you may have seen two different things from -Stree, and yet they are actually both right in terms of the transitive lib set, but wrong in how they are showing you the tree#2020-07-2715:20kenny@alexmiller While that may be happening, the S3 Maven dep approach would result in a classnotfound exception when loading dev-local in the REPL (something with fressian). Switching to local/root with the jar does not result in the classnotfound. I also uploaded a pom to S3 and that approach did not result in a classnotfound.#2020-07-2716:01alexmillerwhat jar are you talking about wrt s3?#2020-07-2716:02alexmillerdev-local?#2020-07-2716:02alexmillerif so, what you're saying makes sense#2020-07-2716:03alexmillerafaik, maven will use the pom to report deps (which is a layer that clj is going through for that)#2020-07-2716:04kennyYes - dev-local.#2020-07-2716:05kennyTo be clear, here's what works and doesn't work.
S3 jar - classnotfound
S3 jar, jar md5 - classnotfound
S3 jar, jar md5, pom - works
local/root jar - works#2020-07-2717:23alexmilleryeah, that makes sense to me. sorry I didn't understand what you were doing earlier#2020-07-2717:23alexmillerlocal/root jar will use the pom inside the jar#2020-07-2803:12olivergeorgeI'm curious how RELEASE resolves in this maven dep...
{:extra-deps {clj-kondo {:mvn/version "RELEASE"}}
:main-opts ["-m" "clj-kondo.main"]}
Perhaps it's a normal maven feature. Hard to google for "release"!#2020-07-2803:26alexmillerRELEASE (and LATEST) are special "virtual" version numbers that tell Maven to look for the newest released version (non snapshot)#2020-07-2803:27alexmillerthey are officially unsupported by clj (but do kind of work with some big caveats)#2020-07-2803:28alexmillerbig caveat #1 - they break caching and you will never see a new release after the one that it first resolves to unless you -Sforce (or otherwise cause a cache recompute)#2020-07-2803:29alexmillerat some point, we may actually prevent caching at all if you use these "versions"#2020-07-2803:30alexmillersubtle caveat #2 - the answer you get is only as good as the metadata index in the repository. for maven or clojars, no worries. if you're using a bespoke s3 repo, then you probably aren't updating the right metadata files and you usually won't see the newest version#2020-07-2803:32alexmillergenerally, I think it's harmless to do this in -Sdeps to grab "latest". it's mildly bad to do it for tools in your deps.edn aliases (like the example here). it's very bad to do it for your project deps.#2020-07-2805:21pmonksDoes there happen to be an equivalent for git/sha coordinates? That could be quite handy…#2020-07-2806:01alexmillerIntentionally not, for the same reasons re caching#2020-07-2806:04pmonksI guess I don’t understand the caching issue. If you resolve “RELEASE” to a concrete version, and that version happens to exist in the cache already, wouldn’t tools.deps use it?#2020-07-2806:05pmonks(and conversely, if it doesn’t, wouldn’t that version be added to the cache, and thereby eligible for future “cache hits”)#2020-07-2806:06alexmillerThe caching is based on time stamps of deps,edn files. If the file never changes (set to RELEASE), the class path is never recomputed, even if newer versions come out#2020-07-2806:06alexmillerCaching of classpath (not lib version)#2020-07-2806:08pmonksAh ok. I assume there’s a good reason for making the deps.edn file the cache key, rather than just recomputing dependencies the classpath every time and having those in the cache?#2020-07-2806:09alexmillerRecomputing deps starts a second JVM so it’s at least a second, usually more depending on io #2020-07-2806:09alexmillerThis is why clj starts faster than lein usually#2020-07-2806:10alexmillerIt also goes to build repeatability#2020-07-2806:11alexmillerOr “clj repeatability”#2020-07-2806:17pmonksOk. I’ll assume there’s a good reason for needing a second JVM to do that (not requesting an explanation, btw, unless you’d like to provide one), and understood full well the performance implications of starting a JVM.#2020-07-2806:41alexmillerit's running a Clojure program with a different classpath the program you're executing#2020-07-2806:43alexmillerspecifically, it's running tools.deps to compute the classpath based on your deps.edn (talking to Maven, maybe S3, maybe Git, etc). in the end it spits out the results in cache. Your actual program is then started with the classpath you intended. Once we've cached that work, we can spit the first part and just run the program.#2020-07-2806:52pmonksSo the advantage of caching based on the deps.edn file, rather than the classpath it contains, is all the I/O to talk to Maven, S3, git, etc.?#2020-07-2807:09alexmillerwell, that's all of the work. not much point to caching otherwise#2020-07-2807:11alexmillerexpanding and downloading a full deps tree can easily mean dozens of jars and many megabytes of network traffic (if your m2 and/or git cache is cold)#2020-07-2807:16alexmillereven if you have all the jars, expanding the tree is more than you'd want to wait, example in tools.deps.alpha repo (~80 jars in the full tree), all jars in cache so no downloads:#2020-07-2807:16alexmilleramac:tools.deps.alpha alex$ time clj -Sforce -e nil
real 0m3.028s
user 0m12.567s
sys 0m0.756s
amac:tools.deps.alpha alex$ time clj -e nil
real 0m0.745s
user 0m1.647s
sys 0m0.133s
#2020-07-2807:17alexmiller(one caveat here is that the clj uber jar is not currently aot compiled - that's coming soon and will improve that first time, which includes a lot of loading)#2020-07-2807:18alexmillerbut it's still not time anyone wants to wait#2020-07-2807:20alexmillerwith a cold m2 cache, time is about 7.5 s (aot may knock a couple seconds off that)#2020-07-2815:00pmonksI can see why that approach will be faster, though it breaks down (cache becomes stale) in the presence of dependencies that are expressed using a “floating” version reference (RELEASE, LATEST, SNAPSHOT, etc.).
Those were deprecated in Maven3, and perhaps tools.deps should follow suit and emit a warning when it encounters such a thing?#2020-07-2815:00alexmillerperhaps#2020-07-2805:24seancorfield@pmonks My dot-clojure deps.edn shows how you can resolve the head of the default branch -- in the comments.#2020-07-2805:25seancorfieldhttps://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L171-L175#2020-07-2805:26seancorfieldAnd then (load-master 'clojure/tools.trace)#2020-07-2805:26seancorfield(following the https://github.com URL pattern, not the org.clojure/tools.trace Maven artifact pattern)#2020-07-2805:41pmonksPotentially dumb question, but is it expected that code inside an EDN file would be executed? Or does that rely on a coincidence of how the Clojure EDN reader functions?#2020-07-2805:53seancorfield@pmonks Can you give a specific example?#2020-07-2805:54seancorfieldI mean, EDN can contain tagged literals and you can provide reader functions that process those... but that's not exactly "code inside an EDN file".#2020-07-2805:58alexmilleredn files are data, not code, and are not evaluated#2020-07-2806:00alexmillerThey are read but that will only construct tagged literals, and clj does not provide a way to install readers for tagged literals#2020-07-2806:03pmonks@seancorfield your example includes code (`(require '[clojure.tools.gitlib :as gitlabs]) …`) - how is that code evaluated?#2020-07-2806:06seancorfieldThose are comments showing what you would execute in your REPL...#2020-07-2806:07seancorfieldThe files contains lots of comments explaining how to use the aliases, with links to repos.#2020-07-2806:09pmonksOh ok. So I would not put those in a deps.edn then.#2020-07-2806:09seancorfieldHuh? Those are comments and they are in a deps.edn file.#2020-07-2806:10pmonksYes they are, and as a result I would expect to uncomment them and have them Just Work:tm: .#2020-07-2806:10seancorfieldI mean, that repo literally is my ~/.clojure/deps.edn file.#2020-07-2806:10pmonksBut if I’m following you, you’re saying that’s not correct - that those lines need to be copypasta’ed into a REPL.#2020-07-2806:11seancorfieldForgive me but I'm a bit shocked you would think code in EDN files would be evaluated...#2020-07-2806:11pmonksI didn’t - that’s why I asked the question.#2020-07-2806:11pmonksI was quite surprised to see commented out code in an EDN file.#2020-07-2806:12pmonks(and by “quite”, I mean “very”…)#2020-07-2806:13pmonksRegardless, @alexmiller separately explained that by design you can’t do an equivalent of RELEASE for git/sha coords in a deps.edn file.#2020-07-2806:13seancorfieldhttps://github.com/practicalli/clojure-deps-edn/blob/master/deps.edn contains comments with lots of shell commands, and even some elisp -- you wouldn't expect any of that to be executed just by reading the EDN file, would you?#2020-07-2806:14pmonksWhy would you assume I’ve read some random deps.edn file? Sadly I don’t have that much disposable time…#2020-07-2806:17seancorfieldWell, you read my file and came to a ... strange conclusion ... so I offered another example deps.edn and wondered what conclusion you would come to from that...#2020-07-2806:18pmonksIf you re-read my original question (https://clojurians.slack.com/archives/C6QH853H8/p1595913713054800), you might see why I was assuming a deps.edn-only solution, and therefore didn’t understand your non-`deps.edn` answer.#2020-07-2806:18pmonksIn general, answering questions people didn’t ask is a wonderful source of confusion. 😉#2020-07-2806:20seancorfieldWith the Clojure CLI/`deps.edn` the "equivalent" to Maven/Leiningen/etc is often "some code that you run" -- which seemed like an acceptable answer to your question.#2020-07-2806:21seancorfieldThat's why I was so puzzled by your follow-up question, as it seemed unrelated to my answer (I was thinking "No, of course reading EDN won't execute code!").#2020-07-2806:22seancorfieldI guess I should add "Here's some code you can run in your REPL" to my deps.edn file where there are code examples 🙂#2020-07-2806:23pmonksThat would have avoided any confusion, no doubt. 😉#2020-07-2806:24seancorfieldIt is sometimes so hard to predict your audience's state of mind 😆 (when writing documentation, I should add).#2020-07-2806:27pmonksProfessional technical writing is such an under-appreciated skill, I’ve found.#2020-07-2807:20spfHey guys. If i have a setup as described in https://clojure.org/guides/deps_and_cli#_using_local_libraries. is there a way to fire up a repl in the hello project which reloads the time-lib deps if they are changed?#2020-07-2807:22alexmillerno, that's not a built-in feature of Clojure (or Java). there are ways to do some variants of this but it's not common#2020-07-2807:22alexmillerwell, do you mean local dir or local jar?#2020-07-2807:23alexmillerif local dir, you can just use Clojure's normal :reload or :reload-all mechanism built into require#2020-07-2807:23spfi meant local dir#2020-07-2807:23alexmilleryeah, so that's just normal reloading#2020-07-2807:23spfok. cool. thx#2020-07-2813:06rickmoynihanany idea why running brew upgrade clojure/tools/clojure installs 1.10.1.561 not 1.10.1.590 ?#2020-07-2813:28alexmiller561 is the newest stable release#2020-07-2813:28alexmiller590 is a dev release#2020-07-2816:10seancorfieldFollowing up from my comment in the announce thread:
{:paths [:clj-paths :resource-paths]
:aliases
{:clj-paths ["src/clj" "src/cljc"]
:resource-paths ["resources"]}}
Is that correct, or should :clj-paths and :resource-paths be underneath an alias?#2020-07-2816:11seancorfieldIf that is correct, I don't see the value since they are fixed labels and values -- am I missing something @alexmiller?#2020-07-2816:12alexmilleroh, that is right - they are aliases#2020-07-2816:12alexmilleran alias is just a name for some data#2020-07-2816:13alexmillerthis was the original meaning here but we have kind of gotten off talking about things that way by tying it into clj's usage of alias data to affect classpath construction#2020-07-2816:14seancorfieldOK, so how is that useful since they are just fixed values?#2020-07-2816:14alexmillerat the moment, it's not very useful#2020-07-2816:14seancorfieldAlso, you have this paragraph twice "Replace project environment ("tool")" -- one with plenty of detail, one with very little.#2020-07-2816:16alexmillerthat's actually intentional - the first part is kind of an overview, and the second is more how the tool works with more detail#2020-07-2816:16alexmillerdoing a lot of heavy editing on this page lately but will likely change more over time#2020-07-2816:17seancorfieldOK, I think it's just really confusing at the moment with two identical headers...#2020-07-2816:17alexmillerI'm ok with that for the moment :)#2020-07-2816:17alexmillerthe toc at the top side bar may make the structure more evident#2020-07-2816:18alexmillerI've gone through about 5 different structures for this page recently, and I don't think I'm done#2020-07-2816:23seancorfieldOn another wordsmithing issue: now that page refers to "tool" as something that might want to override :deps and/or :paths, this section name might be confusing (a different meaning of tools) https://clojure.org/reference/deps_and_cli#_clojure_tools_usage#2020-07-2816:24alexmilleryeah, I've been fighting that, haven't decided what to do about it yet#2020-07-2816:24alexmillernot enough words :)#2020-07-2816:24alexmillerit's like naming things is hard#2020-07-2816:28seancorfieldTools are just so useful 🙂#2020-07-2816:14alexmillerbut if you had say, a build tool that wanted to also use a subset of paths it might be useful#2020-07-2816:14alexmiller;)#2020-07-2816:17seancorfieldThe Maven install example is missing an alias tho' (even if the paths one isn't), yes?
{:aliases
{:fn clojure.tools.deps.alpha.install/install
;; :args map could be provided but can pass on command line instead
}}#2020-07-2816:18seancorfieldThat's meant to be executed with -X:some-alias (most people will pick -X:install I suspect)#2020-07-2816:18alexmillerah, yes that is#2020-07-2816:18alexmillerI actually saw that late last night and forgot to fix#2020-07-2816:20seancorfieldI'm curious as to why that ability surfaced as a built-in option, given that JAR files can already be used as :local/root deps?#2020-07-2816:21alexmillerit's not a built-in option, it's a built-in program#2020-07-2816:22alexmillerdidn't seem big enough to make into its own thing, depended heavily on all the code in tools.deps, wanted it for dev-local, etc#2020-07-2816:22seancorfieldInteresting...#2020-07-2816:22alexmillerserves as a good example :)#2020-07-2816:22alexmillerthe plan is to shift some of the magic switches in clj over to just be programs you can -X#2020-07-2816:23alexmillersome of those might become built-in to the root deps.edn, not sure yet#2020-07-2816:30seancorfieldThe install line you gave in the announcement doesn't seem to work
Error: No available formula with the name "clojure/tools/
#2020-07-2816:30seancorfieldThat's from brew install clojure/tools/#2020-07-2816:30seancorfield(after a brew uninstall clojure)#2020-07-2816:33seancorfieldShould be 1.10.1.590#2020-07-2816:33alexmillerAh, shoot#2020-07-2816:34alexmillerEdited#2020-07-2816:54rickmoynihanwow need to get my head around these new tools.deps features; they might make some stuff I’ve been planning to do completely redundant 🤔#2020-07-2816:58rickmoynihanhow is the -T alias different to having an alias with a :deps key?#2020-07-2816:58alexmillerIt’s not#2020-07-2816:58seancorfieldI need to figure out how to get the clojure script updated on a couple of our older servers since this would help eliminate some of the extra script stuff we've written (but I think there's a blocker for installing clojure on a couple of our old servers).#2020-07-2816:59alexmillerIt is exactly what was already there via -A, just has its own switch now#2020-07-2817:00rickmoynihanok so it just makes the intent clearer on the command line?#2020-07-2817:01alexmillerYep#2020-07-2817:01alexmillerA is meant to be an “all” kind of thing but this didn’t have its own name, so now it does#2020-07-2817:02rickmoynihanso what would -T do with :extra-deps instead of :deps?#2020-07-2817:03alexmiller(Which also helped further clarify some things in the code which has gone through some substantial changes)#2020-07-2817:03alexmillerIt would be ignored#2020-07-2817:04rickmoynihanwow I had no idea how many deps had unqualified names. I thought it was just a handful, but it’s way more than I expected#2020-07-2817:04alexmillerEach of the sub processes takes what it expects and ignores the rest, like all good Clojure programs :)#2020-07-2817:06seancorfieldShould this check include tools_aliases now?
-h|--help|"-?")
if [[ ${#main_aliases[@]} -gt 0 ]] || [[ ${#all_aliases[@]} -gt 0 ]]; then
break
else
help=true
shift
fi
;;#2020-07-2817:08alexmillerNo?#2020-07-2817:09seancorfieldMaybe I'm just missing the actual flow of the script then, nm.#2020-07-2817:10alexmillerYeah this has to do with falsely interpreting a -h appended to main aliases as a help opt#2020-07-2817:10seancorfield(I was just grepping for (tool|all).alias through the new script out of curiosity)#2020-07-2817:11alexmillerWould be curious if you try the win install :)#2020-07-2817:13seancorfield@alexmiller re: -h -- that means that clojure -A:test:runner -h and clojure -T:test:runner -h behave differently.#2020-07-2817:14seancorfield(! 1537)-> clojure -T:test:runner -h|more
Version: 1.10.1.590
...
(! 1538)-> clojure -A:test:runner -h|more
Unknown option: "-h"
Wasn't sure whether that was intentional or not since you said -T and -A were "the same"?#2020-07-2817:15seancorfield(the first case displays help -- -h is not passed to the main program; the second case passes -h to the main program in the test runner here)#2020-07-2817:27alexmillerthis is kind of a corner case - the condition is guarding against having -A/-M supplying the first part of the main args and then -h on the command line where it is not actually the main args to pass to clojure.main. I'm not worried about it.#2020-07-2817:40seancorfieldFair enough. Just wanted to point out that it is a difference between -A and -T 🙂#2020-07-2817:57alexmillerthere's no difference in how :deps and :paths work, which is what I meant#2020-07-2817:24rickmoynihanJust tried doing -X:myfn in one of my projects and I get the error:#2020-07-2817:24rickmoynihanclojure -X:myfn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.
#2020-07-2817:25alexmillerare you on clj 1.10.1.590?#2020-07-2817:26rickmoynihanI thought I was#2020-07-2817:26rickmoynihanyep#2020-07-2817:26rickmoynihan$ clojure --help
Version: 1.10.1.590
...
#2020-07-2817:30seancorfieldI get that too, also on 1.10.1.590:
(! 974)-> clj -Sdescribe
{:version "1.10.1.590"
:config-files ["/usr/local/Cellar/#2020-07-2817:30alexmillerif you clj -Sdescribe and then jar tf <the-install-dir/clojure-tools-1.10.1.590.jar | grep exec.clj#2020-07-2817:30alexmillerdo you see it?#2020-07-2817:32alexmillerwhat installer are you using?#2020-07-2817:32rickmoynihanthe jar is further down than that in /usr/local/Cellar/#2020-07-2817:33alexmillersorry, bad multi copy/paste on my part, that's right#2020-07-2817:33rickmoynihanexec.clj is in there yes#2020-07-2817:33alexmillerthis is mac? linux?#2020-07-2817:33rickmoynihan$ jar tf /usr/local/Cellar/
#2020-07-2817:33rickmoynihanmac#2020-07-2817:33seancorfield(yup, same here)#2020-07-2817:33rickmoynihanbrew was the installer#2020-07-2817:34alexmillerclj -Sdeps '{:aliases {:myfn {:fn clojure.core/pr}}}' -X:myfn is a standalone example#2020-07-2817:34rickmoynihanedn is broken#2020-07-2817:34seancorfieldFails for me#2020-07-2817:35alexmilleredn is broken means?#2020-07-2817:35rickmoynihanah wait sorry was in a project with a broken edn file#2020-07-2817:35seancorfield(! 982)-> clj -Sdeps '{:aliases {:myfn {:fn clojure.core/pr}}}' -X:myfn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.
Full report at:
/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/clojure-12219293780956477884.edn#2020-07-2817:35alexmillerah, I was able to repro#2020-07-2817:35rickmoynihan$ clj -Sdeps '{:aliases {:myfn {:fn clojure.core/pr}}}' -X:myfn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.
Full report at:
/var/folders/ln/536xqskd3_g80n5pdbsjkrjw0000gn/T/clojure-8308868078358895829.edn#2020-07-2817:35rickmoynihantoo#2020-07-2817:35alexmillerit matters whether you're in a directory with a deps.edn#2020-07-2817:36alexmillerwell, that's definitely a bug :)#2020-07-2817:36alexmillerwill fix#2020-07-2817:36alexmillerthx#2020-07-2817:36seancorfieldCrashes for me whether there's a deps.edn file or not.#2020-07-2817:36rickmoynihanyeah same#2020-07-2817:36alexmillerhm, maybe it's a cached classpath for me or something#2020-07-2817:37rickmoynihanthere are 3 hard things in computer science…#2020-07-2817:37seancorfieldI tried it via an alias in a deps.edn file (shown above), then tried via the command line with just {} in deps.edn, and then tried in a directory with no deps.edn -- all of them crash the same way. Even tried -Sforce and it still failed.#2020-07-2817:38rickmoynihanI tried two of those; just tried with -Sforce too — and same#2020-07-2817:38alexmillerwell, that's weird. :) I've been using this for a while and others have tested it as well, somehow never saw an error#2020-07-2817:39alexmillerI'm sure it will make sense when I understand why it's doing it ... this is why it's a dev release!#2020-07-2817:40alexmiller-X has to go at the end, in case that affects anything you tried#2020-07-2817:41rickmoynihannope, I had -X as my only alias#2020-07-2817:42rickmoynihanWell I’d be happy to try and help narrow debug it but I’m afraid I need to call it an evening 😞
I’ll check back here tomorrow.
Thanks for your help, and also thanks for the feature… it looks like it’ll be really useful.#2020-07-2817:43alexmillerah, this makes sense to me now, and why it worked for me#2020-07-2817:44alexmillerand why things others tried worked. such a combination of coincidences... anyhow will fix#2020-07-2817:45seancorfieldNow I'm curious, what's the bug?#2020-07-2817:46alexmillerit's not putting that class on the classpath, obviously :)#2020-07-2817:46alexmillerI was testing in the tools.deps.alpha repository itself#2020-07-2817:46alexmillerwhere it is#2020-07-2817:47alexmillerand others were testing things that used the install program, which requires you to pull in tools.deps.alpha#2020-07-2817:47alexmillerso those all worked#2020-07-2817:47alexmillerI may actually have flubbed a merge somewhere#2020-07-2819:00alexmiller@seancorfield @rickmoynihan ok, new clj 1.10.1.596 is available, should fix this#2020-07-2822:39rickmoynihanworks now! Thanks 🙂#2020-07-2823:10alexmillerthx#2020-07-2819:05seancorfieldLooks fixed! Thank you!#2020-07-2819:08alexmillerif you try it on windows (or linux), would be happy to hear there too#2020-07-2819:09seancorfield@vlaaad If you want to get strings from the command line in your -X-executed function, you can always use *command-line-arg* (although you'd have to discard leading arguments passed to the CLI itself I guess)#2020-07-2819:11seancorfield(! 1010)-> clj -A:test -X:example :test value
("-X:example" ":test" "value") ({:test value})
Looks like you'd only have to skip over the first arg?#2020-07-2819:12vlaaadI don't want to make every arg a string 🙂#2020-07-2819:14seancorfieldHow would you want the CLI to distinguish between "I want this as a string" vs "I want this as an EDN value"?#2020-07-2819:17seancorfieldIt seems like you'd have to try to read as EDN and if you got an error or a symbol then use the whole argument as a string, else whatever the EDN reader produced? Which is open to all sorts of surprises I would expect...?#2020-07-2819:20vlaaadI guess it's impossible to have it both simple and easy 😞#2020-07-2819:24vlaaadI don't want to deal with escaping because I use both powershell and bash that have different escaping rules. With those, whenever I have a need to escape I wish I just used a file. This might be a bit too limiting for convenience, but I'd be fine with this:
everything that can be read as number/boolean/keyword/nil -> number/boolean/keyword/nil
everything else, including things that can be read as collections -> string#2020-07-2819:27vlaaadit doesn't work with repeated args though#2020-07-2819:32vlaaadI've been thinking a bit more about simple vs easy, and currently I think I'm overreacting, I think escaping might be an okay price to pay for the limitations different approaches provide#2020-07-2819:33alexmillerwhen you have static data, the best thing to do is put it in deps.edn and reference via alias#2020-07-2819:34seancorfield> including things that can be read as collections -> string
I think that's a bad idea since it's common to want to pass a map of stuff when calling Clojure function (in general) so that choice would force EDN parsing on everyone#2020-07-2819:35alexmillerwith -X you are always passing a map - the command line is overrides to the data in deps.edn#2020-07-2819:35alexmillerand the overrides support nested paths#2020-07-2819:35seancorfieldRight, I meant as one of the arguments (inside that map)#2020-07-2819:35alexmillernested paths gets you out of another set of cases#2020-07-2819:35seancorfieldi.e., to produce a nested map#2020-07-2819:36vlaaadlike -X:deploy :envs '#{"prod" "qa"}'#2020-07-2819:36seancorfieldYeah, having :test 1234 and :test [1234 5678] produce a number and a string respectively would be very strange.#2020-07-2819:36vlaaadI agree 🙂#2020-07-2819:36alexmillerit turns out vectors are actually ok bash if you use the comma#2020-07-2819:36vlaaadnot ok pwsh#2020-07-2819:37alexmiller:test [1234,5678] is actually ok#2020-07-2819:38seancorfieldI sympathize with the additional pain on PS (I too use it from time to time with the Clojure CLI) but since the vast majority of Clojure CLI users are on macOS/Linux I'd rather not make all their lives more painful, just to make PS less painful -- and it's already a bit painful to use for the Clojure CLI...#2020-07-2819:40alexmillermaybe this is a topic for #clj-on-windows but what's the story on the new Windows terminal?#2020-07-2819:40vlaaadhmm, I might be tripping, [1,2,3] worked fine in pwsh#2020-07-2819:48seancorfieldand clj -X:example [:test,:value] 1234 works on macOS so, yay, portability! 🙂#2020-07-2819:45seancorfield@alexmiller The new Windows terminal is just a fancy tabbed shell around existing PS, cmd, and Linux terminals.#2020-07-2819:45alexmillerso nothing new#2020-07-2819:47nateout of curiosity, why are unqualified lib names now deprecated? was there a deeper issue that needed addressing or is it just because qualified names are unambiguous and more correct?#2020-07-2819:47Darin Douglasshttps://clojurians.slack.com/archives/C06MAR553/p1595944089258600?thread_ts=1595943934.258100&cid=C06MAR553#2020-07-2819:48nateah#2020-07-2819:48nateI saw it in the blog post that he posted https://insideclojure.org/2020/07/28/clj-exec/#2020-07-2819:48natewhich mentioned the deprecation again, but I was curious if there was a deeper reason for it#2020-07-2820:25dominicmTo encourage use of groupid#2020-07-2820:26dominicmAnd discourage hiccup/hiccup in favor of com.weavejester/hiccup. But in a general sense, not the global sense.#2020-07-2820:29sveriFinally 🙂 That was one thing that always made me wonder why one would have unqualified library names when clojure is so heavily rooted in java.#2020-07-2819:52alexmillernot sure what I can add beyond what's in the blog#2020-07-2819:52alexmillerunqualified names are bad so we're going to stop encouraging them in this area#2020-07-2819:53natethat makes sense, thank you#2020-07-2820:17vlaaadPS C:\Users\Vlaaad\Projects\tdeps> cat .\deps.edn
{:aliases {:prn {:fn clojure.core/prn}}}
PS C:\Users\Vlaaad\Projects\tdeps> clj -Sdescribe
{:version "1.10.1.596"
..."}
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.#2020-07-2820:21alexmillerhrm, let me check on that, shouldn't be using that class anymore#2020-07-2820:21vlaaad-Spath is
C:\Users\Vlaaad\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\Vlaaad\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\Vlaaad\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;src
`#2020-07-2820:22alexmillerpath is fine, it's the class, I missed one change there#2020-07-2820:25alexmillerwindows 1.10.1.600 has the fix#2020-07-2820:26dominicmDoes -X compose?#2020-07-2820:28borkdudeWhat do you mean by that?#2020-07-2820:28alexmillerdepends what you want to compose with :)#2020-07-2820:28alexmillerwith other -X, no#2020-07-2820:28alexmillerwith all the classpath stuff, yes#2020-07-2820:29alexmillerbut if you want to run multiple things, write a program that does so and run it with -X :)#2020-07-2820:31dominicmI don't want to write lots of programs! I want to change my programs on a whim at the cli. Start a socket server, start a whatever, run the program in the local project that does setup, run my custom dev function.#2020-07-2820:31dominicm(We've discussed before though, not really trying to convince, but I hoped that exec would provide this)#2020-07-2820:32dominicmRight now I'm (ab)using the fact you can supply an init and a main to load a dev.clj in my home directory, which will run things based on the directory the jvm started in...#2020-07-2820:34alexmillerthere's this thing called a repl that lets you do like ... anything#2020-07-2820:35dominicmYeah, but I have to like type all these weird ( characters.#2020-07-2820:35vlaaadPS C:\Users\Vlaaad\Projects\tdeps> clj -Sdescribe
{:version "1.10.1.600" ...}
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
Execution error (ExceptionInfo) at clj-exec/check-first (clj_exec.clj:32).
Invalid first arg to exec: prn
#2020-07-2820:36vlaaadcould it be PS escaping away the :? 🤔 other args like -A:alias work fine thogh#2020-07-2820:47alexmillerwill take a look, not sure#2020-07-2820:50alexmillermaybe I'm missing the first arg I match in the pwsh for this, but seems like we'd get nothing here, not part of the arg#2020-07-2820:53alexmillerhow hard is it for you to just modify the ClojureTools.psm1 ?#2020-07-2820:54alexmillerI think line 84 just needs to be $ExecAlias += $arg, $params#2020-07-2820:54alexmillerit's losing the matching $arg right now#2020-07-2918:32vlaaadI tried what you suggested, now clj "-X:prn" :a 1 succeeds, while clj -X:prn :a 1 still fails with error Invalid token: :#2020-07-2918:42alexmillerhrm#2020-07-2918:48vlaaadhacked it to work with this:
$sym, $params = $params
$ExecAlias += "$arg$sym", $params
now it doesn't work with "-X:prn" though#2020-07-2918:51vlaaadseems to be PS parsing issue:
> echo -X:prn
-X:
prn
#2020-07-2918:51vlaaad> echo -A:prn
-A:
prn
#2020-07-2918:52vlaaadhmm, that works:
s> clj -A: beep:boop:bap
WARNING: Specified aliases are undeclared: [:beep :boop :bap]
Clojure 1.10.1
user=>
#2020-07-2918:54alexmilleris there a difference in powershell parsing a single thing (into leading char / remaining char) or many things?#2020-07-2918:55vlaaadyou mean -XXX:yyy vs -X:y ?#2020-07-2918:58alexmillerno, I mean are we getting different behavior on clj -X:foo vs clj -X:foo :a 1 ?#2020-07-2919:01vlaaadno:
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn :a 1
{:a 1}
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
nil#2020-07-2919:02vlaaadbut this wont work: clj "-X:prn", because I made the script to expect args -X: and prn separately#2020-07-2919:06vlaaadso what we really need to do is accept both -X:alias and -X: , alias for powershell to make it predictable#2020-07-2919:06vlaaadI'll try to write some powershell..#2020-07-2919:11vlaaadif ($arg -eq "-X:") {
$sym, $params = $params
$ExecAlias += "$arg$sym", $params
} else {
$ExecAlias += $arg, $params
}#2020-07-2919:12vlaaadPS C:\Users\Vlaaad\Projects\tdeps> clj "-X:prn" :a 1
{:a 1}
PS C:\Users\Vlaaad\Projects\tdeps> clj "-X:prn"
nil
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
nil
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn :a 1
{:a 1}#2020-07-2919:24alexmillerthe code for other aliases in tools.deps is actually tolerant of the split here, maybe that's necessary for the -X stuff too. in other words, we've been masking this problem#2020-07-2820:37dominicmIn earnest, there's a little bit of a desire to hide implementation behind flags. Telling someone to run clojure -A:nrepl:piggieback is easier than explaining that they need to run a set of functions once waiting for the repl to connect, and then press buttons on their editor, in an exact sequence to make it work.
Right now I use jvm options.#2020-07-2820:56seancorfieldDang, we use a lot of single-segment lib names at work! 😞#2020-07-2820:57seancorfield(updating 27 deps.edn files in our monorepo!)#2020-07-2821:00alexmillerfeedback on the warning message? I tried to make it useful#2020-07-2821:03seancorfieldIt's perfect.#2020-07-2821:03seancorfieldAnnoying, but perfect.#2020-07-2821:04seancorfieldThe only slightly weird thing was that I seemed to get all of the warnings twice?#2020-07-2821:12alexmillerthat dimly rings a bell. I wrote this like a month ago. I think because it's encountered during canonicalization and also during expansion or something?#2020-07-2820:58alexmillerI have https://www.youtube.com/watch?v=jsW9MlYu31g running through my head ...#2020-07-2821:00seancorfieldAm I right in thinking the new "alias data" feature is going to allow people to add all sorts of configuration stuff into deps.edn as data under an alias?#2020-07-2821:01alexmillerif they like#2020-07-2821:01seancorfieldFolks have been asking for that for quite a while 🙂#2020-07-2821:01alexmillerlike most things, taste is helpful#2020-07-2821:01seancorfield(and have been told in the past, "don't do that" 🙂 )#2020-07-2821:01alexmillerit might be a good place to put like, build configuration#2020-07-2821:02seancorfieldHmm, and because aliases don't compose, it's "just" fixed data so I guess that's a nice place to draw the line.#2020-07-2821:03seancorfieldI suspect shadow-cljs and other tools might leverage it for build config to avoid having a separate file for that...#2020-07-2821:07dominicmAliases can compose, but it's down to their interpreter to decide how.#2020-07-2821:08alexmillerbtw, I don't know if add-libs works with latest (haven't tried). I am still injecting the lib map atm which is what add-libs works starts from, but not sure if it also uses anything else in tools.deps that's changed. the branch I have has the stuff to switch to basis but also other stuff too so some interpolation will need to be done, not sure when I'll get to that#2020-07-2821:09seancorfieldNP. I just updated my dot-clojure deps.edn to rename the alias from :deps to :add-lib and add notes that it should be considered experimental and likely to break or go away 🙂#2020-07-2821:10seancorfield(I had forgotten that the installed deps.edn contains a :deps alias when I originally added it)#2020-07-2821:11seancorfieldI also just cleaned up the :rebl* aliases to reflect the latest REBL having its own deps.edn files for java8 and openjfx15ea (plus some reorganization and updating several versions).#2020-07-2821:11alexmillercool#2020-07-2821:13borkdudeMy work is now finally using deps.edn/tools.deps/clojure CLI (how do you even call this stuff?). I moved out our front-end build to it (using figwheel.main / cljs.main) last week while we still do the uberjar and REPL stuff with boot. Also now we can finally run Java 11 everywhere, since we were stuck, the boot frontend stuff didn't work with it (at least not for us).#2020-07-2821:15ghadithat's exciting @borkdude#2020-07-2821:15alexmillernice!#2020-07-2821:17borkdudeAlso we use this dev script to start 3 processes simultaneously: clojure -A:cljs/dev, clojure -A:less/dev and ./boot dev, while having only one terminal tab open:
https://gist.github.com/borkdude/8f5dff7c2330ca520403eb44c9013a83, just as a convenience#2020-07-2821:20dominicm3 jvms! I bet that doesn't integrate well with cider jack in. Might I suggest starting 3 threads instead?#2020-07-2821:21borkdudeI've always avoided cider-jack-in, I always use cider-connect. Even more so now I'm developing on a remote machine but editing files in my local emacs using tramp.#2020-07-2821:25borkdudeThreads may work but it's not a huge deal right now. We need at least two JVMs given we're using two different build/classpath/dependency tools right now.#2020-07-2821:26seancorfield"I've always avoided cider" -- ftfy 🙂#2020-07-2821:27seancorfieldAs part of this monorepo deps cleanup I'm doing (to prevent all those DEPRECATED warnings!) I'm also removing the last few references to nREPL in our setup since neither I nor my team mate use that now.#2020-07-2821:28dominicmseems unfair to people who use jack in. ¯\(ツ)/¯#2020-07-2821:28borkdudeWell, I asked my teammates and this was all OK for them, so I guess we're good 🙂#2020-07-2822:36rickmoynihanI’m curious why the new tools.deps adds a tool to do local maven installs. Is this something people frequently need?
I’ve probably only ever needed to install free form jars into my local maven repo a few times in the past ten years. I mean sure if I’ve built a jar myself I will frequently want to install it, but it seems strange to supply a method to install jars, but not build them.
I’m just curious what problem it’s solving for people.#2020-07-2822:39rickmoynihanAm I missing something in my workflow; or is it just that this happens to have been baked before other build tasks in tools.build?#2020-07-2822:56kennyPerhaps related to Datomic's dev-local needing to be installed into your local Maven repo to be used (it's not distributable).#2020-07-2823:03seancorfield^ That is what Alex said when asked earlier @rickmoynihan#2020-07-2823:06rickmoynihanahh ok makes sense in that use case. Thanks @seancorfield#2020-07-2823:10andy.fingerhut@alexmiller Robot Chicken has some pretty darned funny riffs on Star Wars scenes, including the one you linked above: https://www.youtube.com/watch?v=WpE_xMRiCLE#2020-07-2901:15seancorfield(OK, running 1.10.1.596 on all our servers at work now 🙂 )#2020-07-2901:50salamhow would one represent this in deps.edn?
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc-bom</artifactId>
<version>19.7.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc10</artifactId>
</dependency>
</dependencies>#2020-07-2901:51salamif possible, of course.#2020-07-2902:00salamfor the context: https://www.oracle.com/database/technologies/maven-central-guide.html#DIY#2020-07-2902:03seancorfield@abdusalam Pretty sure you'll need to figure out what that Bill Of Materials expands to and add each of those dependencies directly to deps.edn#2020-07-2902:06salamthanks, Sean. does it mean there's no direct, built-in support in tools.deps for this kind of use case?#2020-07-2902:25seancorfieldNot as far as I'm aware.#2020-07-2902:35alexmillerno, deps does not support the BOM / import scope stuff#2020-07-2902:38alexmillerit seems radical, but deps just expects you to list the specific dependencies you're using :)#2020-07-2903:03salami see. oracle's jdbc driver (com.oracle.database.jdbc/ojdbc10) used to list other "extra goodies" as dependencies in its pom.xml. i was looking for an elegant way to exclude the extra stuff in order to avoid doing the "exclusion dance" but just found out that it's dropped all of the dependencies since version 19.7.0.0. all we need to do now is to add com.oracle.database.jdbc/ojdbc10 {:mvn/version "19.7.0.0"} to get just the driver itself. 🙂#2020-07-2903:03salamall good now#2020-07-2903:23alexmilleryeah, that's all most people need. the whole bom thing seems like something an enterprise java developer thought up while they were high#2020-07-2906:54rickmoynihanSo I really appreciate the expanded deps guide and reference, but there are two things from the previous iteration of the docs that I found useful that appear to have disappeared now:
One was the old deps data-flow/arg/alias mapping diagram: https://github.com/clojure/clojure-site/blob/master/assets/images/content/guides/deps/deps.png which I appreciate might now either need to be two diagrams, or reworked into a more complex one. Whilst it’s not something I used often, I thought it was effective in the guide at showing what aliases influenced each phase of execution etc.
And the other perhaps more importantly was the “Aliases” section in the old docs here: https://github.com/clojure/clojure-site/blob/285fda270445a70af9b1ec7bdb069309d2d224d7/content/reference/deps_and_cli.adoc#aliases which clearly described how keys in a deps file were used by the command line alias flags.
Are there any plans to bring updated versions of these sections back?#2020-07-2914:05alexmillerthe docs are (always) a work in progress. I pulled the diagram b/c it was way out of date. I have updated it a number of times during the process of the basis changes but I have concerns about whether it actually provides that much value as it has grown increasingly complicated. All of the info from the old Aliases section is still in the doc.#2020-07-2919:50onetomi found that diagram quite helpful, when i was initially learning the clojure cli.
in practice however, i never really had to use anything else then -A so far...#2020-07-2907:02rickmoynihanSecond question — how stable are the new features in tools.deps? In particular the -X feature and the args map data? Are they likely to experience breaking changes between now and the next stable release? I get the feeling not.
Sometimes it’s not clear to me what is blessed as stable and isn’t… i.e. the installer, the command line args, or the tools.deps api. (Though I guess it’s all technically alpha because it’s t.d.a not t.d)#2020-07-2913:09alexmillershould be stable unless we discover some reason to change. not expecting to discover anything but that's why we're letting people try it. I'm expecting the gap to stable to be like days, not a long time.#2020-07-2915:55seancorfieldI was surprised that 1.10.1.590 seems to be the latest stable version on brew already, given the flurry of bug fix releases yesterday (1.10.1.596 and 1.10.1.600).#2020-07-2916:40alexmillerIt shouldn’t be?#2020-07-2916:41alexmillerhttps://github.com/clojure/homebrew-tools/blob/master/Formula/clojure.rb has 561#2020-07-2916:43alexmillerSo I’m curious why you said that#2020-07-2916:43seancorfieldI just did brew upgrade clojure on WSL1 on my Windows laptop last night and it updated me to 590. Hence my surprise.#2020-07-2916:43seancorfieldNote: brew on Linux.#2020-07-2916:43alexmillerThat’s homebrew core, which I no longer update#2020-07-2916:44alexmillerclojure/tools/clojure is official#2020-07-2916:44seancorfieldI know. Hence my surprise 🙂#2020-07-2916:48alexmillerWell I released 590 a week ago, just took me a while to doc and announce#2020-07-2916:48alexmillerBut I guess somebody updated it#2020-07-2910:46thheller@alexmiller how come you are dumping all the new features into :aliases? kinda confusing to give aliases multiple meanings IMHO. maybe I'm just not seeing the bigger picture though.#2020-07-2910:53thhellerlike why is the exec stuff in :aliases and not a new top-level :exec key or so?#2020-07-2911:05borkdudeas far as I have understood it: :aliases was always intended for naming a chunk of EDN data that could be re-used elsewhere, but it's only now that we can see that being used in more places#2020-07-2913:07alexmillercorrect - I would flip it around to say :aliases is a generic facility, and we are making more use of it#2020-07-2911:06borkdudeI wonder if you can combine -A and -X to include extra deps in an -X invocation#2020-07-2913:06alexmilleryou can, absolutely#2020-07-2913:06alexmillerthat's kind of the idea#2020-07-2913:51rickmoynihan🤔 so you can change the function that executed with the default data (by changing the classpath via :extra-deps); but you can’t change the data for a function alias#2020-07-2914:00alexmillerI'm not sure that's "changing the function" as much as supplying the function#2020-07-2914:00alexmillerand you can override the data on the command line#2020-07-2914:01alexmillermaybe I'm not getting what you're describing#2020-07-2914:02rickmoynihanis that intended usage?
It feels a bit weird to have the same fully qualified function name, but with two different definitions (in two different :paths for example).
Would it not be more natural for -X :args to compose?#2020-07-2914:02alexmillerI don't understand why you have that#2020-07-2914:02alexmillercan you more fully describe what you're talking about, because I don't get it#2020-07-2914:10vlaaad@rickmoynihan you mean something like that?
{:aliases {:deploy-to {:fn my.ns/deploy}}
:prod {:args {:url ""}}
:qa {:args {:url ""}}}
clj -X:deploy-to:prod#2020-07-2914:14rickmoynihanyes#2020-07-2914:26vlaaadI thought it would work, but it doesn’t..#2020-07-2914:37alexmilleryeah, that's not a thing#2020-07-2914:40alexmillerI would probably do this:
{:aliases
{:deploy-prod
{:fn my.ns/deploy
:args {:url ""}}
:deploy-qa
{:fn my.ns/deploy
:args {:url ""}}}#2020-07-2914:40alexmillerbut there may be other variants depending on shared config, command-line overrides etc#2020-07-2914:40alexmilleralso my.ns/deploy could interpret data in :args as something to look up in :aliases via the basis#2020-07-2914:41alexmilleryou don't get that for free, but it's like 2 lines of code#2020-07-2914:41alexmillerthese 2 lines https://insideclojure.org/2020/07/28/clj-exec/#runtime-basis#2020-07-2914:55vlaaadYeah, I get what you mean. Basis FTW!#2020-07-2915:59seancorfieldThe basis stuff looks very interesting, but you lose access to it if your build your artifacts as uberjars and run them via java -jar (although I guess you could still do -Dclojure.basis='{...}' 👀 )#2020-07-2916:00seancorfieldIt almost makes me want to go back to running code in production via clojure instead 🙂#2020-07-2914:11rickmoynihanYeah sorry… I think I misinterpreted the motivation behind what @borkdude was saying as a way of effectively doing what @dominicm was asking earlier about composing :args data.
But I misread :extra-deps as being :deps.
Either way it seems :deps would kind of allow the kind of thing @dominicm was asking for; but in a nasty way… i.e. you could swap the function rather than the data. e.g.
- clj -A:myfn -X:args
- clj -A:myfn2 -X:args
Where myfn and myfn2 aliases defined the same ns but with different function definitions.#2020-07-2914:18dominicmI want to run fn2 and fn1 in this example. Tbh, I assumed args and fn had to co exist.#2020-07-2914:20alexmillerI'm unclear if you have a question, and if so, what it is...#2020-07-2914:21dominicmNot a question, I already asked mine!#2020-07-2914:21dominicmJust clarifying that I don't think this would allow for multiple functions like on after :)#2020-07-2914:24alexmillerno#2020-07-2914:11rickmoynihanNot saying this is a good idea btw!!!#2020-07-2914:12alexmillerone helper here is that :args can also refer to an alias instead of a map#2020-07-2914:13vlaaadthat’s something new!#2020-07-2914:12rickmoynihanoh right… that is interesting!#2020-07-2914:12rickmoynihanseems more useful than the :paths one#2020-07-2914:13alexmillerwell the :paths one is foreshadowing :)#2020-07-2914:14alexmillerI did forget to mention that in my blog writeup, but I think it's in the docs#2020-07-2914:14alexmilleror maybe it isn't!#2020-07-2914:15rickmoynihanI don’t recall seeing it#2020-07-2914:15vlaaadyeah, me neither#2020-07-2914:15alexmillerit was in some version of the docs but looks like it got lost in the N reworks I did#2020-07-2914:15rickmoynihan😆 easily done#2020-07-2914:15alexmilleranyhow, that's a thing#2020-07-2914:16rickmoynihanwow — I was silently disappointed that something like this wasn’t there#2020-07-2914:16rickmoynihanbut it’s great that it is#2020-07-2914:16rickmoynihanis there an example of how it works?#2020-07-2914:23alexmiller{:aliases
{:f1 {:fn my/fn1
:args :data}
:f2 {:fn my/fn2
:args :data}
:data {:something :here}}}
and then
clj -X:f1 or clj -X:f2#2020-07-2914:17vlaaadwhere else can I put aliases?!?!?!#2020-07-2914:17alexmillerdata#2020-07-2914:17alexmillerit's just edn#2020-07-2914:18alexmilleroh sorry, misread your question#2020-07-2914:18alexmillerif you read https://insideclojure.org/2020/07/28/clj-exec/ check out the basis injection stuff there#2020-07-2914:27dominicmNow we have runtime basis, does that mean you can go back and interpret the exec aliases however you like?#2020-07-2914:36alexmilleryou can write a program that uses alias data#2020-07-2914:36alexmillerwhich is broader than what you said#2020-07-2916:03seancorfieldI asked in a thread, but I'll ask in the main channel for everyone to see: given that clojure.basis is a property added by the clojure command, what is the thinking behind leveraging this in production code if you normally build uberjars and run those with java -jar? All of that lovely combined deps.edn structure has gone by that point...#2020-07-3009:11rickmoynihanI also had a similar thought when I stumbled across the basis property etc.
Though I figured you could spit out the basis to a clojure.basis file inside the uberjar, and slurp it as an io/resource in your -main.
However the big disadvantage to this is that the basis would be generated at build time rather than run time, so it would be more like a manifest, than what the basis is really supposed to be.
On reflection I think your idea is better — though I’m not entirely sure what you’d use the basis for in that case, other than as a means of using the clojure tool to provide config to your app.#2020-07-3009:21rickmoynihanI guess the only real advantage is using the clojure tool and a deps.edn to manage the classpath in production code, rather than the underlying java command line.#2020-07-3009:25rickmoynihanI have in the past done things like java -cp:myapp-uberjar.jar:some/server/resources/* myapp.main
to for example add assets to an apps resource path as a production overlay… so I guess using the clojure tool itself to help manage these kind of things with aliases might be useful; though I’m not convinced the benefits are huge#2020-07-3009:26rickmoynihanthough I guess it might sometimes be useful to for example provide profiles in production for enabling socket servers etc 🤔#2020-07-3009:30rickmoynihanAlso another usecase is you could provide extra tooling for prod systems in this manner.
e.g. a set of aliases for various production tasks, e.g. triggering a -A:backup alias over cron etc.#2020-07-3009:31rickmoynihanand it would mean you don’t need to waste time inventing new command line parsers and config formats etc#2020-07-2916:04seancorfieldIt almost feels like it's worth switching to using clojure to run uberjars via an alias in the deps.edn so that all that stuff remains available in a "production" style context...#2020-07-2916:04ghadimy guess is that an uberjar tool could convey the basis into the jar#2020-07-2916:05ghadithe more I think about uberjars, the more disadvantages I see vs. sync'ing deps#2020-07-2916:05seancorfieldThe basis differs depending on the aliases used to run clojure tho', right?#2020-07-2916:06seancorfield(because the paths and lib map etc can be different from invocation to invocation -- although the set of aliases etc remains the same)#2020-07-2916:06ghadiI'm shooting from the hip#2020-07-2916:07ghadii'd mostly want the alias data#2020-07-2916:07ghadifor stuff like config#2020-07-2916:08seancorfieldI'd add it to depstar but I think at this point I want to wait and see what tools.build ends up being 🙂#2020-07-2916:11seancorfieldI might experiment with using clojure to run uberjars tho' later today puts on mad scientist hat...#2020-07-2916:11ghadiwdym?#2020-07-2916:12ghadiusing clojure instead of uberjars?#2020-07-2916:12seancorfieldHaving an alias that treats the uberjar as a :local/root dependency and runs the expected -main function inside it, so you get easy access to the t.d.a. / Clojure CLI infrastructure#2020-07-2916:19seancorfieldThis:
:aliases
{:run {:deps {worldsingles/worldsingles {:local/root "../../build/uberjars/worldsingles-1.0.0.jar"}}
:fn worldsingles.publisher/-main}}
...
And then clojure -X:run#2020-07-2916:20seancorfieldSo you get the "benefit" of an AOT'd uberjar with the benefits of the Clojure CLI. And the program can read the basis 🙂#2020-07-2916:23seancorfield(of course, now you need the Clojure CLI installed on a target server, not just the JVM, and you need a deps.edn file in the folder where you start the app)#2020-07-2916:35vlaaadAnd internet access#2020-07-2916:35vlaaadAnd credentials for private repos#2020-07-2916:37seancorfield@vlaaad Not if the entire :deps is just the uberjar, right?#2020-07-2916:37seancorfieldThere are no other deps to fetch at that point.#2020-07-2916:40vlaaadAh, sorry, misread the situation#2020-07-2916:41vlaaadI was thinking about using clj without uberjar#2020-07-2916:42seancorfieldRight, which we used to do on production (and, on one server, for "reasons" we still do that).#2020-07-2918:31borkdudeNot that I really care, just wanted to report that if you call a private function with -X, it also works:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "1e7c716c757a6b7a7b5e535c4e2c2e2f27"}, :content ("[email protected]")}#2020-07-2918:44alexmillerthat's not intentional but interesting :)#2020-07-2918:44alexmillerit does make sense given the impl#2020-07-2918:46seancorfieldBecause requiring-resolve can get at private Vars and there's no access check on that dynamic call path, right?#2020-07-2918:48alexmillerright, just like calling through a var#2020-07-3121:09borkdudeThe private var call still works with 619 🙂#2020-07-3121:16alexmillerI'm ok with that#2020-07-3121:16alexmillerI don't consider it a bug#2020-07-3121:16alexmillerprivate functions are only in your mind#2020-07-3121:17borkdudeI'm going to quote you on that when somebody will ask questions about this 😂
$ bb '(ns foo) (defn- foo [] :foo!) (ns bar (:require foo)) (foo/foo)'
:foo!
#2020-07-2918:40p14nANybody else having issues with the recent tools-deps docker?#2020-07-2918:42p14nSpecifically
Caused by: Syntax error compiling . at (aleph/http/core.clj:251:3).
Caused by: java.lang.IllegalArgumentException: No matching method setTransferEncodingChunked found taking 2 args for class io.netty.handler.codec.http.HttpUtil
#2020-07-2918:42p14n(I'm making assumptions here because nothing else has changed)#2020-07-2918:44seancorfield@p14n change in JDK version perhaps? Last night I was helping some beginners debugging a weird reflection-based problem that changed between JDK 11 and JDK 14 (it was broken on 8 and 11 but worked on 14).#2020-07-2918:46p14nThat's what I thought. I switched to openjdk-8-tools-deps (tools-deps now 11) but no dice yet#2020-07-2919:12p14nFixed by using clojure:openjdk-8-tools-deps-1.10.1.502 :shrug:#2020-07-2919:13seancorfieldA change in the default Clojure version perhaps, with stricter syntax checking?#2020-07-2919:14ghadii don't use any of the clojure:* images#2020-07-2919:15ghadithey are not made by clojure team, and are not necessary IMHO#2020-07-2919:15ghadiI use an adoptopenjdk image and then add the clojure installer#2020-07-2919:19p14nDo you host your own?#2020-07-2919:15cap10morganHow is the groupId derived in clojure -Spom? mine keeps getting reset to the artifactId when I run it.#2020-07-2919:16seancorfieldThat's a bug. That should not happen.#2020-07-2919:16seancorfield(and it's a new bug ^ @alexmiller)#2020-07-2919:17cap10morganah, interesting. this is with 1.10.1.590.#2020-07-2919:17alexmillerI don't think anything has changed there#2020-07-2919:17seancorfield(! 1133)-> head pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation=" ">
<modelVersion>4.0.0</modelVersion>
<groupId>seancorfield</groupId>
<artifactId>next.jdbc</artifactId>
<version>1.1.569</version>
<name>next.jdbc</name>
<description>The next generation of clojure.java.jdbc: a new low-level Clojure wrapper for JDBC-based access to databases.</description>
<url></url>
<licenses>
Wed Jul 29 12:17:19
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1134)-> clojure -Spom
Wed Jul 29 12:17:30
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1135)-> git diff
diff --git a/pom.xml b/pom.xml
index 3dce875..5632254 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation=" ">
<modelVersion>4.0.0</modelVersion>
- <groupId>seancorfield</groupId>
+ <groupId>next.jdbc</groupId>
<artifactId>next.jdbc</artifactId>
<version>1.1.569</version>
<name>next.jdbc</name>
#2020-07-2919:18seancorfieldThe only change it makes is to overwrite the groupId#2020-07-2919:17alexmillerwell, nothing intentional :)#2020-07-2919:21alexmillercan you file a jira, I'm juggling many conversations right now#2020-07-2919:22cap10morganWill do#2020-07-2919:25cap10morgan@seancorfield The expected behavior was just that it should leave that value alone when the pom.xml already exists?#2020-07-2919:28cap10morganHopefully that captures it: https://clojure.atlassian.net/browse/TDEPS-157#2020-07-2919:31souenzzoThe function that turns (merge-cli-ops {:default :opts} "[:extra :opts]" "true") => {:extra {:opts true} :default :opts} is public?#2020-07-2919:42alexmillersorry, I don't understand the question#2020-07-2919:46alexmilleror how it relates to tools.deps#2020-07-2919:50souenzzoI think that is this one
https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/clj_exec.clj#L59#2020-07-2919:51souenzzoI liked the "DSL" to "assoc-in" values.#2020-07-2920:47alexmillerOh, yes that’s it. Treat anything about this code as implementation detail though#2020-07-2920:47alexmillerIt may migrate into Clojure itself#2020-07-2919:39seancorfield@cap10morgan Yup, it should only touch the dependencies in an existing pom.xml file.#2020-07-2919:43alexmillerwell, it's broader than that - deps, repos, directories#2020-07-2919:44alexmillerthere has been some work done internally here to make this more configurable and that will eventually make its way into user configurability#2020-07-3002:14cflemingIs there anything in the new tools.deps stuff that would make removing the Clojure coming from the system deps file easier? My main use cases are using a fork of Clojure, and also using Deps to manage non-Clojure projects (i.e. parts of Cursive).#2020-07-3002:16seancorfield@cfleming Can't you just use :override-deps and an alias?#2020-07-3002:16cflemingI don’t think that works.#2020-07-3002:17cflemingI’ll play around with that and see, but IIRC it didn’t work last I tried.#2020-07-3002:17seancorfieldAs for non-Clojure stuff, nothing yet but maybe tools.build might fingers crossed ?#2020-07-3002:17cflemingtools.build?#2020-07-3002:18seancorfieldI know you can select any released Clojure version via :override-deps so I assume you could point it to a forked artifact...?#2020-07-3002:18seancorfieldtools.build was mentioned in one of Alex's blog posts with a "shhh, secret" comment and it's popped up occasionally here#2020-07-3002:19seancorfieldI assume it's something Cognitect have developed internally, probably to help with Datomic (like the recent -X option in the Clojure CLI with the install option for Datomic dev-local).#2020-07-3002:19cflemingMaybe I will finally be able to get off Ant 🙂#2020-07-3002:21cfleming@seancorfield Looking at the docs, :override-deps seems to only be about overriding versions, not changing the artifact.#2020-07-3002:22seancorfieldI assumed your "artifact" would have the same group/artifact ID but the "version" would point to :local/root (a JAR)?#2020-07-3002:27cflemingWow, what do you know - it works!
~/d/tools-deps-test> cat deps.edn
{:override-deps {org.clojure/clojure {:local/root "/Users/colin/.m2/repository/com/cursive-ide/clojure/1.10.1/clojure-1.10.1.jar"}}}⏎ ~/d/tools-deps-test> clj -Spath
/Users/colin/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:src#2020-07-3002:28cflemingThanks @seancorfield, that may make my life much easier.#2020-07-3002:29cflemingFor modules where I don’t want Clojure, I should be able to use that trick to point Clojure to an empty jar, and then just filter that out when building.#2020-07-3002:42seancorfieldYou just made my week! 🙂#2020-07-3002:43cflemingAnd mine 🙂#2020-07-3002:44cflemingOops, wait a minute - I didn’t look closely enough. The classpath still refers to the original Clojure 😞#2020-07-3002:44seancorfieldAw, sorry...#2020-07-3002:46cflemingActually, no, the root cause is that I’m an idiot. Here’s the working version, with override-deps in an alias as it should be:
~/d/tools-deps-test> cat deps.edn
{:aliases {:fork {:override-deps {org.clojure/clojure {:local/root "/Users/colin/.m2/repository/com/cursive-ide/clojure/1.10.1/clojure-1.10.1.jar"}}}}}
~/d/tools-deps-test> clj -Spath -A:fork
/Users/colin/.m2/repository/com/cursive-ide/clojure/1.10.1/clojure-1.10.1.jar:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:src#2020-07-3002:47seancorfieldOh, thank goodness!!#2020-07-3016:36dominicmOne risk of the new runtime basis is that programs are hard-coded against them, e.g. a repeat of the lein-figwheel problem. So the clj tools become mandatory for tooling. Setting a system property is easier than whatever assumptions lein plugins make about the environment. We should perhaps dissuade ourselves from doing this 🙂#2020-07-3016:37dominicmOn another note, if I do store information inside of deps.edn (e.g. a figwheel build info), how could I reload it? Is that part of tools.build?#2020-07-3016:41hiredmanthe add-lib branch distinguishes between the initial basis (static set at launch) and the runtime basis mutable as you add libraries#2020-07-3016:41hiredmanso I imagine you don't reload the basis#2020-07-3016:41hiredmanyou take it as an initial value and stick it in an atom and mutate that#2020-07-3016:47seancorfieldThere are definitely tradeoffs involved. When we got started with Clojure a decade ago, we used lein run in production and continued to use that approach when we switched to Boot (2015) and later to the Clojure CLI. It's only been since that switch that we started building uberjars instead I think, and for a long while, we did not AOT code for those. In the end, we added the AOT step purely to improve startup time (it made a big difference for a couple of our processes, so they could come back into the cluster faster after a deployment). We've always generally had our app configuration in external files -- separate from the build/run tooling -- and/or via system properties (or environment variables). But I don't have a concern about requiring our programs to be run via clojure if there are enough benefits to be worth switching from java -jar or java -cp.#2020-07-3016:47hiredman(add-lib2 I guess, add-lib still calls it the lib-map instead of the basis)#2020-07-3016:49seancorfieldAs for "clj tools become mandatory" -- we're sort of already in that place: to work with a given project pretty much requires you use whatever tooling that project has adopted, be it lein, boot, or clj.#2020-07-3016:57alexmillerthe basis is just a jvm property that points to a file that is an edn map. right now, clj makes that and injects it but nothing there is "special". lein could make a basis map and pass it too. or a user could make it and set it in jvm opts in lein.#2020-07-3016:59dominicm> we're sort of already in that place:
That's true for projects, but not tools. To clarify, I mean that your project makes clojure cli mandatory (although it could support lein alongside clj too, some projects do this). I'm worried about tooling like lein-figwheel or lein-sass, etc. which is tied to a particular JVM start tool needlessly.#2020-07-3016:59alexmiller@dominicm rather than expecting to reload the basis, you could have the basis include a reference to a different file that you watch and reload (or whatever)#2020-07-3017:00alexmillertooling that is tied to a particular tool (ie a lein plugin) was always not a good factoring of the problem. it is far better to write a tool that is independent and just code and then adapt it to tooling as needed (and to prefer tools that levy minimal requirements such that the adapter may be nothing)#2020-07-3017:01alexmillersome lein plugins are written well for this, some aren't#2020-07-3017:02dominicmThe clojure tools have inherently encouraged a lot of this through encouraging adoption of clojure main, so there's an inherent reduction of coupling to only be Clojure itself for many tools. (Not mine, mine all read deps.edn files!)#2020-07-3017:02alexmillerI don't know enough about lein-figwheel or lein-sass or whatever to say on those (but having "lein" in the name seems like a flag)#2020-07-3017:04dominicmOne consideration will be what is standard/non-standard in the runtime basis. Lein can't tell you what aliases were loaded, it's a semantic mismatch. It could tell you some information about the deps though.#2020-07-3017:15alexmillerhttps://clojure.github.io/tools.deps.alpha/#clojure.tools.deps.alpha/calc-basis#2020-07-3017:16alexmillerdocs the basis map return (more detail is needed, a bit of it is intentionally a little vague right now, but it's mostly pretty set). I have not yet updated the specs for tools.deps but I will get to it eventually#2020-07-3017:17dominicmTo me, those seem like they should be namespaced to tools deps if you think it's OK for other tools to generate. With the exception of lib-map, which I could see being generated by other tools (boot/lein)#2020-07-3017:29seancorfieldThe overall map is the combined deps.edn hash map, with those four extra keys added, so you're already tied to the deps.edn structure in the clojure.basis property. Leiningen isn't going to be able to create that master-edn map anyway, unless it adopts t.d.a. and deps.edn as an option (alongside project.clj) and changes its dependency resolution approach. So it seems you're already tied to clojure.basis and those simple keys at this point -- no need to qualify them, IMO.#2020-07-3017:30seancorfieldPerhaps Leiningen could/should produce a lein.basis file/property when running code, that reflects the hash map structure of the project.clj file? But I don't think you can expect portability between tools (and I don't think you should).#2020-07-3017:33seancorfield"Lein can't tell you what aliases were loaded, it's a semantic mismatch." -- clojure.basis doesn't tell you what aliases were used to run the program, only the resulting "basis" of those aliases, but it includes the whole merged master-edn so you get "all" aliases available -- and lein could provide the :aliases structure from project.clj (although it's a completely different beast).#2020-07-3022:25pmonksSo interesting finding just now that might be useful to others. I operate a tiny little chat bot, that runs on a cheap / low-spec PAAS. Earlier today I switched from running it via the Clojure CLI (i.e. via something like clojure -m mybot.main) to constructing an uberjar and running that (i.e. via something like java -jar mybot.jar), mostly in order to reduce startup times (the PAAS doesn’t provide blue/green deployment at the subscription level I’m on, and the brief outage during scheduled restarts had been noticed and commented on by some users).
As expected, startup time basically went to zero. What I didn’t expect was how much of a memory saving there’d be too. Previously, my bot had stabilised at around 340MB total (heap & off-heap) memory usage, which was quite close to the 512MB total provided by the PAAS, and which has to cover all processes’ memory use (not just that of my bot’s JVM).
Since the switch to an uberjar, total memory usage has stabilised at around 210MB. While I didn’t confirm this in detail, my preliminary conclusion is that the extra ~130MB is allocated by the tools.deps machinery (after all that’s the only thing that changed), and is not released after tools.deps has done its thing and is no longer in use.#2020-07-3022:37alexmillerThose are separate jvms so I don’t think I buy that if that was your theory#2020-07-3022:37pmonksWhat is your theory?#2020-07-3022:38pmonksNote: the uberjar is not AOT compiled (except for the ns with the -main function).#2020-07-3023:09hiredmanaot compilation is transitive, so unless you work to avoid it, aot compiling a single namespace will also aot all namespaces it depends on as well#2020-07-3023:14pmonksIs that true of (gen-class) ?#2020-07-3023:15pmonksThat’s the only thing I’m using (since it’s required in order to generate an “executable JAR” that java -jar can run directly).#2020-07-3023:15pmonksThat said, this is my first use of depstar, so perhaps it’s doing something like that…#2020-07-3023:20hiredmanit is complicated, but if you are using gen-class in the ns form, then yes, everything is being aot compiled#2020-07-3023:22pmonksI think you’re onto something. I cracked open the uberjar and sure enough my code is in there twice in .clj and .class form.#2020-07-3022:39alexmillerI would buy it for a container around the clj call, not sure what the PAAS is or how it works#2020-07-3022:40pmonksHeroku dyno (“hobby” subscription level). It’s pretty simple - a VM, as best I can tell. I’m not using Docker or any other container-like deployment mechanism.#2020-07-3022:40alexmillerI guess I'm saying "is allocated by the `tools.deps` machinery" is not in your final process, but may consume resources in the container itself#2020-07-3022:40pmonksDoes that JVM stick around though? Once the “app” JVM is started?#2020-07-3022:40alexmillerno#2020-07-3022:41alexmillerthe script waits for it to exit#2020-07-3022:41alexmillerbefore running the app#2020-07-3022:48alexmillerthe clojure script forks a child process to run the first jvm, waits for it to exit, then exec's the app which replaces the script with the final jvm process#2020-07-3022:48pmonksI’m out of ideas. I did look into whether the local “ephemeral” filesystem might be a RAM drive that counts towards the VM’s RAM total, but that’s not the case (the ephemeral filesystem is 6GB).#2020-07-3022:49pmonks(since my app’s dependencies total around 90MB - somewhat close to the 130MB delta)#2020-07-3022:49alexmillerI can easily envision a container expanding its memory footprint and thus taking more time in the first case or something like that#2020-07-3022:50alexmillerbut certainly the time savings makes sense#2020-07-3022:50pmonksRight, but the dashboard I’m looking at shows total memory used within the VM (and doesn’t give me any idea of the total RAM of the physical hardware that VM is running on).#2020-07-3022:51pmonksYeah, and startup time savings was the impetus for making this change. I’m just surprised (and wish I could explain) the additional savings in RAM usage. Especially given how substantial those savings were (memory use dropped by ~40%).#2020-07-3022:55pmonksHere’s what I see, fwiw, spanning the time of the deployment where I switched to an uberjar:#2020-07-3022:56pmonksThat graph is basically flat, going back a month or two (to when I first deployed the bot).#2020-07-3022:56alexmillerwell, that could be reserved memory based on the max used (affected by the first jvm)#2020-07-3022:57pmonksIt isn’t though - that dashboard further breaks down the JVM’s heap and off-heap memory usage, and both of those have dropped by approximately the same %ages.#2020-07-3022:57alexmillerI don't really have any way to tell#2020-07-3022:57pmonksI wouldn’t expect Heroku to “keep counting” a JVM that’s been terminated, indefinitely.#2020-07-3022:58pmonksLet me see if I can repro locally…#2020-07-3023:05pmonksDifferent numbers obvs, but I think I see a similar pattern locally (using visualvm to look at just heap & metaspace usage, so definitely not an apples-to-apples comparison with Heroku).#2020-07-3023:05pmonksHere’s clojure -m mybot.main:#2020-07-3023:05pmonksAnd here’s java -jar mybot.jar:#2020-07-3023:06pmonks(look at “used”, specifically - the JVM has allocated different total amounts from the OS in these runs, which throws off the vertical scales between the two runs)#2020-07-3023:08pmonksFor easier reading, here are the key numbers:
| clj | jar |
------------+--------+--------+
Heap | 83MB | 39MB |
Metaspace | 71MB | 60MB |
------------+--------+--------+
#2020-07-3023:12pmonks(similar pattern, albeit different numbers, on successive runs, and with a GC requested before taking measurements)#2020-07-3023:13pmonksNo clue what’s going, tbh. I’m not one to look a gift horse in the mouth, but then unexpected changes of this magnitude also make me suspicious. 😉#2020-07-3023:15alexmillerone difference is that the jvm memory maps the jars - and here you've got many jars vs 1#2020-07-3023:15alexmillermight be some per-jar overhead you're avoiding?#2020-07-3023:16alexmilleranother thing to try would be to use -Scp#2020-07-3023:16alexmillerclojure -Scp the.jar -m your.namespace#2020-07-3023:17alexmillerthat would be still using the script, not computing a classpath but using 1 jar#2020-07-3023:18alexmillerguessing that's still the lower number#2020-07-3023:18alexmillerand the next test would be to get the script out of the way. grab the computed classpath with -Spath, then java -jar vs java -cp with that#2020-07-3023:37pmonksYeah I was wondering about total # of open file handles, but it’s hard to come up with 130MB that way. 😉#2020-07-3023:38pmonks@hiredman found something though - it looks like all my code is being AOT compiled (the uberjar contains both .clj and .class versions of all of my namespaces). I don’t know why that’s happening (this is my first time using depstar, so I may have messed something up there), though apparently (gen-class) (which is used on my “-main” ns) can cause transitive AOT compilation too.#2020-07-3023:41pmonksWould runtime compilation of Clojure generate a lot of long-lived memory usage like that though? That’s the bit I can’t quite wrap my head around - this extra memory isn’t being GCed.#2020-07-3023:42pmonks(as part of trying to keep memory usage down on this restricted runtime environment, the bot calls (System/gc) every hour - I know that doesn’t guarantee that the GC will kick in but I figure it can’t hurt, especially as this particular bot is 99% batch/offline functions so it doesn’t matter if it’s not highly responsive to user activity)#2020-07-3023:49hiredmanhave you set a max memory on the jvm?#2020-07-3023:49pmonksWhen running locally, yes: -Xmx300m (I keep it low to try to emulate the memory-limited Heroku environment).#2020-07-3023:50pmonksOn Heroku it’s container managed: -XX:+UseContainerSupport#2020-07-3023:50pmonks(locally my JVM is 11.0.2+9, while Heroku is JVM 11.0.8, btw)#2020-07-3100:23ghadi(isn't container support the default now in jdk11?)#2020-07-3100:31seancorfield@pmonks "I don’t know why that’s happening (this is my first time using depstar, so I may have messed something up there)" -- no, that's just how AOT works. In order to compile your main namespace, it has to load (compile) all the namespaces it depends on so you get pretty much your whole program compiled to .class files. That's just normal. What (:gen-class) does is flag that particular namespace to be generated as a class with methods -- functions beginning with - (by default) will become named methods on that class, instead of (what Clojure usually does which is) to compile each function to its own class with an .invoke() method. So your -main function becomes a public static method of the class (which is what Java expects).#2020-07-3100:33pmonksI didn’t realise AOT-compiled-Clojure couldn’t “thunk” into on-demand-compiled-Clojure.#2020-07-3100:33seancorfieldLeiningen tries to do some fancy post-AOT clean up I think, to delete a lot of .class files that aren't directly part of your project so you don't get "everything" compiled? Can't remember whether that is the default or an option.#2020-07-3100:33pmonks(I almost never use AOT, so have minimal experience with it)#2020-07-3100:33pmonksYeah and all of my previous experience has been with leiningen.#2020-07-3100:34seancorfieldIf you don't want other nses compiled, don't :require them in your main ns -- instead use runtime require/`resolve` (or the nice new requiring-resolve in 1.10).#2020-07-3100:34pmonksThanks - looking now…#2020-07-3100:34pmonksI don’t particularly care either way, tbh. Have just heard folks here recommend against AOTing much, if any, of a codebase (if possible).#2020-07-3100:35seancorfieldOne good test to run here would be to build your uberjar without -C and -m on depstar -- which would not AOT anything -- and then run it with java -cp path/to/the.jar clojure.main -m mybot.main#2020-07-3100:36seancorfieldThat will do compile-on-demand and if it behaves the same as the original clj memory usage, then compile-on-demand might be responsible for the larger memory usage.#2020-07-3100:38pmonksOk now that my 5pm US-PT (actually midnight UTC) batch job is done, I’ll give that a try. 😉#2020-07-3100:42pmonksSo locally, removing AOT seems to have resumed the “higher memory usage” scenario.#2020-07-3100:42pmonksNext up, heroku env.#2020-07-3100:45seancorfieldSo it sounds like there's some overhead from loading source and compiling on demand that doesn't go away? I'm a bit surprised it doesn't all get GC'd eventually.#2020-07-3100:47pmonksYeah me too. Though I only have up to 24 hours of continuous operation available to look at, since Heroku forcibly restarts the dyno (VM) at least once per day. (as mentioned above, that includes at least 24 calls to (System/gc)).#2020-07-3100:49pmonksDeployed. Now we wait for a bit for it to show up on the Heroku dashboard.#2020-07-3100:51seancorfieldI can't say I noticed a difference at work when we switched from compile-on-demand to AOT, but then our smallest process runs with 1G heap so a few 10s of MBs wouldn't really be noticeable...#2020-07-3100:53pmonksMakes sense. I can definitely see the CPU cost of on-demand compilation at startup showing up. 😉 Memory graph still a bit inconclusive…#2020-07-3100:54hiredmanCalling System/gc is silly. Predicting what effect (if any)calling that method has is hard, so calling it, and then trying to reason from the basis of calling it is not helpful#2020-07-3100:54pmonksSure, but it can’t hurt in a non-interactive app.#2020-07-3100:54hiredmanYou should look at the gc logs, and look at the sizes of the generationd#2020-07-3100:55pmonksAnd even a multi-second pause in this bot is preferable to exceeding memory (which puts me in Heroku “R14” jail…).#2020-07-3100:55ghadihow will you be in jail if the memory is bounded?#2020-07-3100:55hiredmanCalling System/gc will never stop you from running out of memory#2020-07-3100:56hiredmanThat is how automatic garbage collection works#2020-07-3100:56pmonksBy exceeding the 512MB quota for everything inside the VM (JVM heap + JVM metaspace + memory required by any other processes + OS memory requirements (caches, etc.) + …).#2020-07-3100:56hiredmanIt can only monkey with the heuristics in the gc#2020-07-3100:57pmonksRegardless, I tested it first (obvs) and it improved memory utilisation over longer periods of time.#2020-07-3100:57hiredmanYou should set a lower max heap for the jvm then#2020-07-3100:57pmonksI don’t set the heap - the container does.#2020-07-3100:57pmonksRemember I’m running with -XX:+UseContainerSupport.#2020-07-3100:57hiredmanThen have the container do a better job#2020-07-3100:57pmonksNot to mention that controlling the heap doesn’t help with metaspace anyway.#2020-07-3100:57hiredmanIf you are exceeding the limits#2020-07-3100:57pmonks(which is where on-demand compiled classes etc. go)#2020-07-3100:58pmonksKnow anyone who works at Heroku to improve their VM impl.? Cause I don’t. 😉#2020-07-3100:58pmonksIt’s not a container.#2020-07-3100:58pmonksIt’s a “dyno”.#2020-07-3100:58pmonksI don’t provide anything but the stuff that gets fed to the JVM.#2020-07-3100:59pmonks(and while Heroku also supports Docker, I have negative interest in switching to that model of app construction)#2020-07-3100:59pmonksSo it’s still a bit early to be sure, but it looks like it was AOT that was saving much of the memory:#2020-07-3101:00pmonks(that second “hump” is when I deployed with AOT disabled)#2020-07-3101:01hiredmanYou should check the different generations, compilation may result in extra old generation garbage, that is never collected because it never needs to be#2020-07-3101:03pmonksFrom what I’m seeing most of the savings are in the metaspace, not the heap.#2020-07-3101:03pmonks(recalling that old generation is on-heap)#2020-07-3101:05pmonksWhich makes sense if on-demand compiled Clojure code is somehow more off-heap-expensive (e.g. metaspace, code-cache) than AOT compiled Clojure code.#2020-07-3101:05seancorfieldIn your original numbers, you only showed 11MB difference in metaspace, but 44MB in heap.#2020-07-3101:05pmonksYeah that’s local.#2020-07-3101:05ghadiif you can AOT, AOT#2020-07-3101:06seancorfieldAye, AOT as the last step of an application, prior to production deployment seems like a reasonable step to me.#2020-07-3101:06pmonks@ghadi I probably will, given the memory restrictions in this environment. That doesn’t help me understand why this is happening though.#2020-07-3101:06pmonksTo date I’ve never used AOT, and if there is a substantial (~40% memory saving, for this one bot) saving to be had, that could be a discriminating factor in choosing AOT vs on-demand compilation for other apps in future.#2020-07-3101:06seancorfieldYou'll need VisualVM or something similar to see what's actually going on inside the heap/metaspace...#2020-07-3101:07pmonks@seancorfield no easy way to hook that up to Heroku, though it runs an agent that does break down on-heap vs off-heap memory usage for their dashboard.#2020-07-3101:07seancorfieldRight, I meant locally, with a low heap size to vaguely mirror what's happening on Heroku.#2020-07-3101:09pmonksYeah - I’m doing that, but the absolute numbers are pretty different than what I see on Heroku. Possibly because it’s hard to limit the JVM’s off-heap memory usage (-Xmx only affects the heap), so it’s happily slurping up the (copious, compared to Heroku) memory on my laptop.#2020-07-3101:11pmonksBTW, here’s a more complete view of heap vs off-heap usage reported by heroku. The two deployments (the first to AOT uberjar, then back to non-AOT uberjar) are pretty obvious:#2020-07-3101:11pmonksStand by - I accidentally chopped off “total memory usage”.#2020-07-3101:11ghadiwe're in the wrong channel, btw#2020-07-3101:12pmonksWe are now, yes. Originally this looked like tools.deps issue.#2020-07-3101:12ghadiI concur with @hiredman’s suggestion to get better details about heap regions.#2020-07-3101:13ghadithere are several interesting metrics depending on the GC used (G1 by default on 11)#2020-07-3101:13pmonksHere’s a better picture:#2020-07-3101:15pmonksEven at this granularity, it’s pretty clear that the delta in the non-heap memory usage is greater than that on-heap.#2020-07-3101:15pmonksi.e. looking at the heap regions likely won’t help#2020-07-3101:16pmonks(and yes the default Heroku dashboard sucks - it’s deliberately setup to encourage one to pay for a better one)#2020-07-3101:18pmonksSadly I have to step away from this now, but it looks like AOT is causing the difference, even if it’s unclear why. Thanks for indulging me on this!#2020-07-3101:31ghadinot a mystery to me. compiling is more work than not compiling#2020-07-3102:06pmonksMore work == CPU load, sure. But that work, once complete, shouldn’t leave long-lived garbage hanging around.#2020-07-3102:08pmonksWell, unless the JVM does tricks for AOT-compiled classes that it can’t use for dynamically generated (non-disk-resident) classes (e.g. mmap’ing pre-compiled .class files rather than keeping them resident in memory at all times, unloading disk-resident .class files once they’ve been JITted, etc.).#2020-07-3102:28pmonksOr if the Clojure compiler has caches that don’t get primed if it’s not called upon to dynamically compile some Clojure code.#2020-07-3120:57alexmillerA new clj release candidate (what we were formerly calling "dev" releases) is now available - 1.10.1.619 (see https://github.com/clojure/homebrew-tools#version-archive-tool-releases for installation info)
• Fixes -Spom regression in overwriting groupId in existing pom.xml files
• Improvements in error handling for -X
• New: -F execution specifier to invoke an arbitrary function that takes a map at the command line: clj -Fclojure.core/pr :a 1 :b 2 => {:a 1 :b 2}#2020-07-3121:03borkdudeAwesome, I already wondered why one had to specify a function in deps.edn instead of just being able to call any function#2020-07-3121:05seancorfield(! 1038)-> clojure -Sdeps '{:aliases {:foo {:fn clojure.core/prn :args "test"}}}' -X:foo
Invalid :args for exec, must be map or alias keyword: "test"
(! 1039)-> clojure -Sdeps '{:aliases {:foo {:fn clojure.core/prn :args :bar} :bar "test"}}' -X:foo
"test"
🙂#2020-07-3121:08seancorfieldThanks for the quick fix to pom.xml groupId!#2020-07-3121:13borkdudeWhy not just pass the entire map as one cmd line arg?
$ clojure -Fclojure.core/pr '{:a 1 :b 2}'
Key is missing value: {:a 1 :b 2}
or support both?#2020-07-3121:14borkdudenot a big deal, but if the map is coming from a file, this could get a bit weird#2020-07-3121:14alexmillerbecause quoting sucks and it's more of a match to the -X#2020-07-3121:15borkdudequoting sucks, but I bet you will soon be quoting the individual args as well, there is no escape 😉#2020-07-3121:15alexmillerbut I implemented both at different points :)#2020-07-3121:15alexmillerno pun intended :)#2020-07-3121:45vlaaadpowershell's -X:alias still broken 😞#2020-07-3121:46vlaaadhave you missed my suggested fix that handles splits to -X: and alias ?
if ($arg -eq "-X:") {
$sym, $params = $params
$ExecAlias += "$arg$sym", $params
} else {
$ExecAlias += $arg, $params
}#2020-07-3121:57alexmilleryeah, I haven't had a chance to look at that yet#2020-07-3121:57alexmillerit's in the queue!#2020-07-3122:38seancorfield@alexmiller I'm a bit surprised this first version fails (in 1.10.1.619):
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "bac9dfdbd4d9fafeffe9f1eef5ea97ebef88eff08bf4"}, :content ("[email protected]")}#2020-07-3122:42seancorfieldIt's not a big deal but it worked before and I wasn't sure if it was a deliberate change or just an artifact of the implementation.#2020-07-3122:49alexmillerYeah, that should probably work #2020-08-0304:08cflemingWith the new :paths alias data feature, I’m assuming that those aliases are just fished directly out of the :aliases map in the (combined system/user/project) deps.edn map? i.e. there’s no alias selection done based on which aliases are passed by the user, or anything like that.#2020-08-0304:44alexmillerRight#2020-08-0304:50cflemingOne further question: Cursive also shows aliases that the user can select in a tool window, which allows the user to decide which source paths etc should be synced to their project. I assume it doesn’t make sense to show these aliases there, i.e. it only makes sense to show those aliases whose value is a map.#2020-08-0304:50cfleminghttps://cursive-ide.com/userguide/deps.html#working-with-aliases#2020-08-0312:07alexmillerSeems right#2020-08-0304:09seancorfield@cfleming That's my understanding based on playing with it a bit.#2020-08-0304:10cflemingThanks @seancorfield. It gets a little confusing in the case of :extra-paths - those are calculated using the provided aliases, but then their alias values are taken from the raw map (I think).#2020-08-0304:46alexmillerIn the latter question, aliases should be resolved from the merged edn map, same as paths. But maybe I’m not getting some nuance of your question#2020-08-0304:46cflemingYep, that was what I had assumed.#2020-08-0304:47cflemingSo using this map:
{:paths ["src" :clj-paths :resource-paths]
:aliases {:clj-paths ["src2"]
:resource-paths ["resources"]
:moar {:extra-paths ["src3" :moar-paths]}
:moar-paths ["src4"]}}#2020-08-0304:48cflemingsrc, src2 and resources will always be selected. src3 and src4 will be selected if the user has chosen to use the moar alias.#2020-08-0304:51seancorfield@U064X3EF3 It feels like this is part of a bigger picture that we don't have all the pieces of -- since the alias references in ;paths etc is all static at the moment, so you can't actually override any of it yet?#2020-08-0312:03alexmiller@cfleming right#2020-08-0312:03alexmiller@seancorfield not sure what you mean about overriding#2020-08-0312:04alexmillerThis is about giving names to sunsets of paths#2020-08-0312:04alexmillerSubsets#2020-08-0312:05alexmillerOne important place these names show up is in the classpath data in the basis#2020-08-0316:54seancorfieldBy "override" I meant that project aliases override user-level, and you can further override those on the command-line -- so I was thinking that you could provide a new :clj-paths alias on the command-line to "override" a :clj-paths alias in the project deps.edn.#2020-08-0316:54seancorfieldLike I say, I'm still not seeing much benefit in that feature yet, so I assume there's a "bigger picture" and it will make more sense "soon" 🙂#2020-08-0317:22alexmilleryou could do all that but I do not have any use case in mind where that would be a good idea#2020-08-0317:22alexmillerbetter to manipulate with a -C alias modifying :extra-paths or overrides#2020-08-0317:46seancorfieldAye, and that's what I would tend to do -- so, again, I'm not seeing much benefit in being able to specify parts of paths etc as aliases right now.#2020-08-0317:54alexmillerthen you should not do it :)#2020-08-0304:11seancorfieldI guess you can supply new values via the command-line (`-Sdeps`) but I don't see much value in it as it stands.#2020-08-0304:12seancorfieldI assume it's part of a bigger picture that we don't have all the pieces of yet...#2020-08-0307:40borkdudeWhat does -S stand for? Just a random name?#2020-08-0311:58Janne SauvalaBased on all the comments I guess it stands for “something” 😁#2020-08-0312:07alexmillerSetting#2020-08-0312:10borkdudeRiiight#2020-08-0312:30alexmillerbut "something" or "stuff" is pretty good too :)#2020-08-0312:54borkdude🙂#2020-08-0312:54borkdudeBtw, I have wondered how you prevent :) turning into an emoticon. Is that a setting?#2020-08-0312:55borkdudeNm, I found it :)#2020-08-0308:08vlaaadI just went through deps guide and reference searching for words starting with "s". My guess is it means "Some option" or "Specify"#2020-08-0311:08dominicmI think it was script#2020-08-0312:08alexmillerSetting#2020-08-0619:15dharriganI have a question, related to this extra-deps, using a couple of :local/root declarations, i.e., one that loads in rebel readline (with little customisation) and another which launches a nrepl (with some middleware injected). Can I still benefit from what the guide says? Can I compile the extra-deps includes, so that my clj startup is reduced?#2020-08-0619:16alexmillersure#2020-08-0619:16alexmillerit's all on the same classpath#2020-08-0619:17dharriganOkay, but where would the classes directory live, i.e., if I have something like this...#2020-08-0619:17dharrigan:rebel {:extra-deps {local-rebel {:local/root "/home/david/.clojure/libs/local.rebel" :deps/manifest :deps}
local-nrepl {:local/root "/home/david/.clojure/libs/local.nrepl" :deps/manifest :deps}}
:main-opts ["-m" "local.rebel.main"]}#2020-08-0619:17alexmillerI'd put it in its own alias and then use multiple aliases#2020-08-0619:17alexmillerie, it's actually the same as what's in the doc, you just might have extra tool aliases in play#2020-08-0619:18dharriganokay, let me have a play around. My clj startup at the moment is taking about 8 seconds to launch...#2020-08-0619:29souenzzoOh....
Mine is ~2min on good machines
and we really don't care about that#2020-08-0619:36dharrigan:shocked_face_with_exploding_head:#2020-08-0619:39souenzzoI open the repl less then once a day.
(it remains open)#2020-08-0619:18alexmillerso clj -A:dev:rebel or whatever#2020-08-0619:18alexmilleror actually you've got main args in there#2020-08-0619:18alexmillerso you might want to do clj -R:dev:rebel when doing the compilation#2020-08-0619:19alexmillerso you get just the deps but not the main stuff#2020-08-0619:19dharriganokay, will have a play 🙂 Thank you Alex! 🙂#2020-08-0619:19alexmiller🍺#2020-08-0619:19dharriganI'd love one thanks 🙂#2020-08-0619:38seancorfieldI only restart my REPL about once a week so even a few minutes startup wouldn't really bother me. I'm curious about workflows where startup times of eight seconds is problematic @dharrigan?#2020-08-0619:40dharriganI have a need, a need for speed 🙂 I guess I'm just learning about if by applying some of the recommendations in that web page I can shave off some time! No harm in trying to optimise if I can 🙂#2020-08-0619:45dharriganInteresting, got it down to 3 seconds 🙂#2020-08-0619:59alexmillerI'll send you my bill#2020-08-0619:59dharriganErrrr....would an I.O.U suffice?#2020-08-0620:00dharriganI.O.U one beer, fresh from the tap, if when you get to Ireland/UK 🙂#2020-08-0620:01alexmillerI hope I can leave my house again someday#2020-08-0620:23lread@dharrigan’s avatar is a duck… he already has a bill!#2020-08-0620:26dharriganHe's the Egyptian Household God of Frustration.#2020-08-0620:30lreadTIL!: https://www.youtube.com/watch?v=im29S6ZWRDI#2020-08-0620:33dharriganBest Show Evar!#2020-08-0620:25seancorfield@alexmiller So I went and tried that dev startup time stuff (for the first time). I compiled a bunch of big libraries that my code relies on that rarely change, but did not compile my own code, then a ran a "load file" on my app's main file and it threw an exception about a clojure.core.cache.CacheProtocol not found... so I'm guessing there's a possibility for this pre-compile step to mess up in some way?#2020-08-0620:26alexmillerdepends how you compiled everything#2020-08-0620:26alexmillerif you're trying to compile independent libs then you might not be compiling in the same order you actually load them in via your app#2020-08-0620:27alexmillerand then you can get into the case of recompiling a protocol, which can definitely go wrong#2020-08-0620:27seancorfieldWhen I tried to repro with a (require 'api.main) instead of the load-file that the editor does, that seemed to work, so maybe it's something specific to load-file?#2020-08-0620:28alexmillerrequire will load too under the hood, but maybe#2020-08-0620:29seancorfieldAh, no difference. I started a fresh REPL, with those libraries precompiled and require failed this time, just as load-file had before.#2020-08-0620:30seancorfieldSo, you have to transitively compile "everything" to make this work, so that compile order doesn't matter?#2020-08-0620:30alexmillertransitively compiling everything should work (b/c it's exactly what your app does when it loads)#2020-08-0620:30seancorfieldWhat about a project where you have multiple entry point nses, that represent separate applications?#2020-08-0620:31alexmillerother things might work :)#2020-08-0620:31alexmilleryou could use more than one alias/classes dir if you find that to be a problem#2020-08-0620:32alexmilleror you could still use just compile with one and use from the other entry point. you might miss some stuff but presumably it's mostly the same in deps.#2020-08-0620:32seancorfieldCurrently, I start a REPL once a week (roughly) and just load code into it as needed so I usually end up with our entire code base loaded into the REPL eventually.#2020-08-0620:33alexmilleryou are a bit outside the target user of this page :)#2020-08-0620:33seancorfieldCompiling two different entry points that overlap might fall into the "recompiling a protocol" area?#2020-08-0620:33alexmillerdepends#2020-08-0620:34alexmillerI am shorthanding the actual problem (recompilation itself is not the actual problem)#2020-08-0620:35seancorfieldSo the problematic piece (that surprises me a bit) is that (compile 'clojure.core.memoize) does not seem to transitively compile clojure.core.cache which I would have expected... I think that's the root of my earlier problem perhaps?#2020-08-0620:36alexmillerhttps://clojure.atlassian.net/browse/CLJ-1544#2020-08-0620:36alexmillerit should transitively compile anything loaded by clojure.core.memoize#2020-08-0620:36alexmillerbut not everything in core.cache is used by memoize#2020-08-0620:39seancorfieldWhen I look in the classes tree after (compile 'clojure.core.memoize) there are no core.cache classes at all.#2020-08-0620:41hiredmanwhat about if you compile clojure.core.cache? my kind of guess would be it is already compiled/loaded somewhere else in your repl#2020-08-0620:41alexmillerany chance you've already loaded something that has loaded core.cache prior?#2020-08-0620:41alexmillercompile piggybacks load so if it's not being loaded (b/c it already is), it won't be compiled#2020-08-0620:43seancorfieldAh, that is a possibility, because something in my dev tooling might well be using that... If I explicitly (compile 'clojure.core.cache) I do see the .class files produced for it which would suggest it isn't loaded previously?#2020-08-0620:43seancorfield(or am I misunderstanding your point?)#2020-08-0620:44hiredmanthat is interesting, I wouldn't expect to see those if clojure.core.cache was previously loaded#2020-08-0620:45alexmillerI suspect you're getting the equivalent of :reload at the top level namespace you compile#2020-08-0620:45alexmillerbut not on the ones below#2020-08-0620:45seancorfieldYup, confirmed. A "bare" REPL with just my dev tooling loaded has clojure.core.cache clojure.core.protocols clojure.core.server clojure.core.specs.alpha clojure.data.priority-map ... (non-`clojure.*` stuff omitted).#2020-08-0620:46hiredmanah, right, compile calls load-one directly, bypassing the already loaded check#2020-08-0620:47seancorfieldSo, yeah, I'd need to explicitly (compile 'clojure.core.cache) to make all this work, right?#2020-08-0620:48alexmilleryou can do what's in the bottom of that guide (kind of), which is to address a similar problem (when user.clj has already loaded stuff)#2020-08-0620:48alexmiller(binding [*compile-files* true] (require 'clojure.core.memoize :reload-all))
#2020-08-0620:49alexmillerthe only thing that makes load do compile is that dynvar#2020-08-0620:49alexmillerand the :reload-all will force a reload#2020-08-0620:56seancorfieldI'm going to stick a list of (compile 'x.y.z) forms in a compile.clj file so I can do load-file on it to compile "everything" that I want pre-compiled and I'll add to it as/when I run into problems. FWIW, just with the set of libs I've put in that list so far, a cold (require 'api.main) has dropped from 18 seconds to 10 seconds. Like I say, since I only restart my REPL occasionally, that's really not a concern but now I'm curious about this idea of pre-compiling dependencies that rarely change...#2020-08-0620:56seancorfield...and if a dependency does change, presumably you'd have to force a recompile of the newer version (and everything that depended on it) in order to pick those changes up?#2020-08-0620:57alexmilleryou don't have to#2020-08-0620:57alexmillera newer dependency should have newer .clj files and be preferred#2020-08-0620:57alexmillerit will just be slower till you recompile#2020-08-0620:57alexmillerclojure takes the newer .class or .clj#2020-08-0620:58alexmillerI guess if you changed deps to a newer version of a lib that was still older than your local .class files that wouldn't have that effect#2020-08-0620:59alexmillerI spent a half day trying to build this into clj so it could do it automatically (since it knows when your deps go stale). it was tricky for a bunch of reasons but maybe a future feature#2020-08-0621:00alexmillerthe point where your .cpcache is stale is also the point when you want to recompile#2020-08-0621:00alexmillerso was trying to hook that#2020-08-0621:00seancorfieldYeah, discussions about it being a possible future feature of clj was what made me even curious to try it, once it was brought up here.#2020-08-0621:03alexmillerrelevant for tools that replace the classpath too#2020-08-0920:11borkdudeI get a ton of "DEPRECATED libs must be qualified" warnings not from my own deps.edn, but for transitive deps. Can I somehow turn that off? It's not something I can fix in my own codebase.#2020-08-0920:12borkdudeAlso one thought: would it make sense to only deprecate it for git deps and not for mvn deps, since mvn already has the foo = foo/foo implicit convention and there's not much tools.deps is going to change about that?#2020-08-0920:23vlaaadThis is not about foo=foo/foo convention, this about a way of thinking about dependencies. Shorthands incentivize names squatting, full group/artifact names make you think who you depend on. I'd rather depend on borkdude/clj-kondo than on just clj-kondo. I'd rather released vlaaad/cljfx than just cljfx... I think it makes sense to state group/artifact for every procurer.#2020-08-0920:23dominicmI don't think maven does have that implicit thing, does it?#2020-08-0920:23dominicmThey're very big on groups#2020-08-0920:34borkdudeIf you push a library foobar with lein to clojars, it's really foobar/foobar, so the group is foobar and the library is also called foobar. That's what I mean with a mvn library. The mechanism for pushing/retrieving is based on mvn. I know avoiding conflicts is important, but this convention has been there since ages in leiningen. Is this a lein convention or a mvn convention, I'm not sure. I haven't used pom.xml directly that much.#2020-08-0920:36borkdudeSo is this a design mistake of lein that deps now tries to solve? I totally understand the need for git deps to have a unique and predictable identifier.#2020-08-0920:36dominicmI suspect it's a lein and clojars thing. Maven central won't let you create an artifact like that, only clojars does (to controversy)#2020-08-0920:40borkdudeAnyway, I hope that I turn off the warning for transitive deps, since I'm getting a ton of them:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "771518051c13021312373a35274547464e"}, :content ("[email protected]")}#2020-08-0920:44seancorfield@borkdude You're only getting those for git deps -- not for all transitive deps.#2020-08-0920:44seancorfieldIf you were depending on released versions, you wouldn't get those warnings. I've only had to update my current project deps to get rid of warnings -- but I don't generally depend on git deps.#2020-08-0920:46borkdudeI'm getting warnings for 1) my own git deps, 2) my own mvn deps, 3) mvn deps of transitive git deps. I'm not sure if a git dep depending on another git dep would also yield a warning, but I suspect this situation isn't that prevalent yet (let's call this 4).
I understand the need for 1. 2 is disputable, but from a "let's fix what lein did wrong" perspective, I can live with it. 3 and 4 I wouldn't find acceptable.#2020-08-0920:47borkdude1 and 2 aren't shown in the output, I already fixed those.#2020-08-0920:49alexmillerYou are getting these any time clj reads any deps manifest#2020-08-0920:50alexmillerSo would for 4 as well#2020-08-0920:52alexmillerI could tailor these more, but this is a step towards removing support entirely#2020-08-0920:56alexmillerThe thing I’m really trying to get rid of here is the need for canonicalization of libs when reading deps files#2020-08-0920:56seancorfieldI have run across a handful of "single" named Java libs (`wsdl4j` plus a few others), but it's pretty rare in non-Clojure world. I agree that we've been very sloppy about that practice in the past (and I'm guilty a few times -- but I'm working to fix that 🙂 )#2020-08-0920:56borkdudeWell, leaving it as it is now will probably force people to update their deps.edns quicker ;)#2020-08-0920:57alexmillerWell that’s the idea :)#2020-08-0921:01dominicmI'm glad there's more of a push on this. I've seen a few conflicts in libraries (I've seen at least two libraries using roll.core).
I've gotten friction in the past because it's "verbose", so it's nice this will become non optional.#2020-08-0921:02borkdudeI think that example just shows how .core doesn't contribute to avoiding conflicts while still having a multi-segmented name#2020-08-0921:04seancorfieldI think .core is a horrible default 😐#2020-08-0921:06seancorfieldThat's why clj-new doesn't do that. You must provide either a group-id/artifact-id project name (and the README talks about using reverse domain names or other identity-based groups) or you can provide a multi-segment namespace name. But I may downplay that option a bit more, now that t.d.a. is "encouraging" proper group/artifact lib names 🙂#2020-08-0923:28cfleming@alexmiller -Stree seems to be broken since 1.10.1.624. It’s because previously root deps used to return nil in their :dependents set, and now they return an empty collection. This broke Cursive, I’ve fixed it there by inlining make-tree and changing:
(let [{roots false, nonroots true} (group-by #(-> % val :dependents boolean) lib-map)]
to:
(let [{roots false, nonroots true} (group-by #(-> % val :dependents empty? not boolean) lib-map)]#2020-08-0923:30cflemingI’m not sure what the cause of that change is, I couldn’t see anything immediately suspicious in the changelog but I only had a fairly cursory look.#2020-08-0923:32cflemingOn a related note, is the :dependents/`:children` info returned from resolve-deps considered to be part of the API and thus something I can rely on? It’s not mentioned in the docstring.#2020-08-0923:32cflemingOn a related note, is the :dependents/`:children` info returned from resolve-deps considered to be part of the API and thus something I can rely on? It’s not mentioned in the docstring.#2020-08-0923:43seancorfieldThe output of -Stree has certainly changed -- I just updated from .619 to .641 and my outdated deps script broke (so that'll be a bit of fun for me to debug and fix tomorrow morning when I get to work!).#2020-08-0923:43seancorfieldThe output of -Stree has certainly changed -- I just updated from .619 to .641 and my outdated deps script broke (so that'll be a bit of fun for me to debug and fix tomorrow morning when I get to work!).#2020-08-0923:43cflemingWell, it’s changed from appearing to not appearing at all.#2020-08-0923:44seancorfieldAh, yes, that would definitely break my script. I hadn't actually tested basic output yet.#2020-08-0923:44cfleming~/d/c/issues (master)> clj -Stree
~/d/c/issues (master)>
#2020-08-0923:45seancorfieldOops.#2020-08-0923:45seancorfieldWell... I guess I don't need to debug my script 🙂#2020-08-0923:46cfleminghttps://clojure.atlassian.net/browse/TDEPS-161#2020-08-1001:47alexmillerI actually rewrote most of the tree stuff this morning, still working on it though#2020-08-1002:24alexmiller@cfleming ah, I see where I broke it, will fix. I don't know who's updating the homebrew core tap, but it's just random versions now#2020-08-1002:26cfleming@alexmiller Ok, I basically copied your make-tree function and fixed my copy in Cursive. As long as I can rely on the :dependents/`:children` bit coming back from resolve-deps that’s probably a better idea anyway, since I won’t have to fix anything if you change make-tree to do something different.#2020-08-1002:30alexmillerthe bug was in the last refactoring of lib-paths#2020-08-1002:31alexmillerso basically in the guts of resolve-deps#2020-08-1002:33alexmilleryou can't rely on :children - that's implementation detail leaking. :dependents you can rely on as that's part of the lib-map data structure and is spec'ed https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/specs.clj#L60-L64#2020-08-1002:34cflemingIs dependents enough to build the tree without children?#2020-08-1002:34alexmillerdepends what you're trying to do with it#2020-08-1002:35alexmillermake-tree is going to be moving and changing quite a bit - are you using that directly?#2020-08-1002:35cflemingBasically, show the user a dependency tree in Cursive.#2020-08-1002:35cflemingI was, but after working around this bug this morning I’m now using a fixed copy of the old version.#2020-08-1002:36alexmillerthis whole thing with you depending on whatever arbitrary version the user happens to have is crazy talk man#2020-08-1002:37cflemingThat’s why I want data I can depend on in the resolve-deps results (or anywhere else, don’t care where I get it from) that I can use to build a tree to show the user.#2020-08-1002:37alexmillerthe reason I'm rewriting make-tree is I suspect the same reason you're relying on that stuff - to better form the tree and the lib-map not being sufficient#2020-08-1002:38cflemingBasically, until there is a defined API that gives me what I need I have no choice but to hack the guts.#2020-08-1002:38cflemingWhich is intrinsically tied to concrete versions.#2020-08-1002:38alexmillerwell, depending on specific concrete versions is fine :) it's having no control over what you're using that seems like crazy talk#2020-08-1002:39cflemingI can never control what the user has installed.#2020-08-1002:39cflemingI can let them choose the version that they want to use, but I can’t mandate versions.#2020-08-1002:39seancorfield@cfleming What are you actually using this information for?#2020-08-1002:39alexmillerI guess I don't see why you can't use a specific version of tools.deps independent from whatever clj they have installed#2020-08-1002:40cflemingBecause when you introduce new stuff in deps, people aren’t going to like having to update Cursive to get it.#2020-08-1002:40alexmillerseems better than having it break every time I update clj#2020-08-1002:40cflemingAnd as long as there’s a stable API, it shouldn’t matter.#2020-08-1002:40alexmillerthere's not a stable API#2020-08-1002:40alexmillerthat's the alpha part#2020-08-1002:41cflemingThen it will break.#2020-08-1002:42cfleming@seancorfield In this particular case, to show the user the deps tree in the IDE: https://cursive-ide.com/userguide/deps.html#viewing-your-project-dependencies#2020-08-1002:45cflemingIf the user finds a particular version of deps breaks with a particular version of Cursive, they can choose the version of deps they want to use within Cursive, or they can downgrade their CLI until either Cursive or deps gets fixed.#2020-08-1002:46seancorfieldI'm continually impressed at the work you do with Cursive but at the same time it just reinforces why I like a simple editor and simple command line tools 🙂#2020-08-1002:46seancorfieldI'm continually impressed at the work you do with Cursive but at the same time it just reinforces why I like a simple editor and simple command line tools 🙂#2020-08-1002:46cfleminged will always be there for you.#2020-08-1002:46cfleming🙂#2020-08-1002:48seancorfieldYou can pry Atom/Chlorine and the Clojure CLI from my cold, dead hands mate!#2020-08-1002:49seancorfield(although I've recently taken to using Reveal instead of REBL, just as an experiment with my workflow... but I'm missing several of the REBL features)#2020-08-1002:50cflemingTo be fair, I think you have a broken script until the CLI is fixed.#2020-08-1002:50alexmillerfixed in clj 1.10.1.645#2020-08-1002:50alexmillerwhich is uploading right now#2020-08-1002:52seancorfieldlooks fixed -- thanks @alexmiller#2020-08-1002:55seancorfieldAnd my outdated deps script works again -- yay!#2020-08-1003:00alexmiller@cfleming I'll give you a heads up before I publish the tree stuff - still working on it.#2020-08-1003:01cflemingOk, thanks.#2020-08-1003:01alexmillerjust as background, deps expansion does not actually ever build an explicit tree during expansion and in fact it's not a tree, it's really a graph.#2020-08-1003:02alexmillerthe make-tree/print-tree stuff is kind of retroactively trying to build a tree from what's in the lib map, but that's really also not an explicit tree and does not contain sufficient path info to respect the exclusions the user originally stated#2020-08-1003:03alexmillerI've rebuilt the tree stuff to work from the trace (which is actually returned in the metadata on the lib map if it was requested), but it's surprisingly complicated to rebuild a tree that matches the user's original intent#2020-08-1003:04cflemingI can believe that.#2020-08-1003:04cflemingMostly I’d just like a stable way to replicate what you show the user on the command line in Cursive.#2020-08-1003:05alexmilleryeah, and there will be a function somewhere that can work from the trace (which you can actually emit from clj with -Strace) to give you that tree data#2020-08-1003:05cflemingI mean, I can actually show them a graph if required.#2020-08-1003:05alexmillerthat's kind of the stuff I'm wrestling with - there are multiple things you might want to show#2020-08-1003:06alexmillerwhat most of the deps tools show is basically a tree with every lib listed once (even though it's really possibly a dep at many points)#2020-08-1003:07alexmillerbut you in particular might really want the "full" tree - full expansions at every node (could still stop at cycles in a path)#2020-08-1003:08alexmillerand then for debugging you really want to know - what version was requested, what version was actually selected, and what libs were excluded at which different points#2020-08-1003:08cflemingYup, and then I could collapse duplicates to mirror what users are used to seeing in other tools, while still allowing them to expand them if required.#2020-08-1003:09alexmillerall of that info is in the version map included in the trace but it requires a fair amount of trawling#2020-08-1003:10cflemingWhen you talk about the trace, do you mean something included in the new basis stuff?#2020-08-1003:11alexmillerno, pre-dates that actually#2020-08-1003:11alexmillerclj -Strace will dump a trace.edn file#2020-08-1003:12cflemingIs that documented anywhere? It doesn’t seem to be in the reference doc.#2020-08-1003:12alexmillerno, I wasn't sure how stable it was going to be. I added that when I made tools.deps.graph so basically prior to my conj talk#2020-08-1003:13alexmillerbut it hasn't actually changed since then I don't think and this tree work will add another "user" of it#2020-08-1003:14cflemingOk, looks like I could use that if the generation of it were available in t.d.a and I could call that.#2020-08-1003:15alexmillerit has been available in t.d.a since since last November in the return of resolve-deps#2020-08-1003:15alexmillerif you pass the :trace option, it will be included in the metadata of the lib map that's returned#2020-08-1003:16alexmillerthat part is doc'ed at https://clojure.github.io/tools.deps.alpha/#clojure.tools.deps.alpha/resolve-deps#2020-08-1003:16cflemingOk. If that’ll be a more stable thing for me to use I’ll start using that.#2020-08-1003:16alexmillerno plans to change it ;)#2020-08-1003:17cflemingWhat’s missing from the doc is descriptions of what’s actually in the trace log, or the runtime basis for that matter. Are they specced or something?#2020-08-1003:17alexmillerno, I wasn't quite ready to commit to them yet and haven't really caught up on that#2020-08-1003:17alexmillerthe basis is doc'ed in calc-basis#2020-08-1003:18alexmillerhttps://clojure.github.io/tools.deps.alpha/#clojure.tools.deps.alpha/calc-basis#2020-08-1003:18alexmillerit is all a bit strewn around atm and specs are missing some of those#2020-08-1003:19alexmillerso, definitely some doc debt to pay back. my time is kind of split right now with joining nubank :)#2020-08-1003:20cflemingOk, when I get some time I’ll investigate switching to that, and falling back to what I do now if the metadata isn’t available. I’ll be interested to see your tree code when it’s done.#2020-08-1011:22pezAnyone know of a docker image containing both leiningen and clojure? I’m migrating a project piece by piece and am using the lein-tools-deps plugin. Currently using jdk 8, as well, unsure how hard that dependency is though.#2020-08-1011:22pezAnyone know of a docker image containing both leiningen and clojure? I’m migrating a project piece by piece and am using the lein-tools-deps plugin. Currently using jdk 8, as well, unsure how hard that dependency is though.#2020-08-1418:33cap10morganThe clojure:latest image now has both lein and CLI tools installed if that helps.#2020-08-1011:37borkdude@pez FWIW I usually just install clojure in a Docker container myself#2020-08-1011:37borkdudeSee any of my latest projects#2020-08-1418:32cap10morganIs this a bug? When I update my pom.xml file with clojure -Spom using http://1.10.1.itit is nesting another <build> element under the existing one. So my diff looks like this:
<build>
- <sourceDirectory>src</sourceDirectory>
+ <build>
+ <sourceDirectory>src</sourceDirectory>
+ </build>
</build>
#2020-08-1418:33alexmillerit is a bug#2020-08-1418:33alexmillerhas been fixed in a subsequent prerelease but not yet in a stable one#2020-08-1418:34cap10morganah ok. cool, thanks!#2020-08-1418:34alexmillerit's fixed as of 1.10.1.590 (latest prerelease is 1.10.1.645)#2020-08-1418:35alexmilleror you could fall back to older stable#2020-08-1418:35seancorfieldWhich prerelease introduced the bug where it overwrote the groupId? (since fixed)#2020-08-1418:36cap10morganexactly what I was wondering 🙂#2020-08-1418:37alexmillerI think it was 1.10.1.590#2020-08-1418:37alexmillerwhich was fixed in 1.10.1.636#2020-08-1418:37alexmillerit is hard to keep track of all these across the two repos#2020-08-1418:38alexmillerbut I would say if you're going to use any prerelease, I would use the latest#2020-08-1418:40alexmillerI have 3 or 4 pending things to get into clj and then I'm hoping to get to another stable release#2020-08-1612:07borkdude@seancorfield Would it be possible to exclude clojure and clojure.spec.alpha sources/AOT-ed assets from an uberjar? The use case would be for babashka which already has clojure built-in (spec not yet, but maybe coming). So an uberjar with only (non-clojure) source files and no AOT-ed stuff would be leaner.
https://github.com/borkdude/babashka/issues/535#issuecomment-674518153
Since you're reading from a system property here https://github.com/seancorfield/depstar/blob/b7e9e4537e33bbf78fbdbc0ef56d9675cb494bc2/src/hf/depstar/uberjar.clj#L283
it might be a small tweak to override the classpath with a command line arg or file reference (java itself uses the @ convention for that).#2020-08-1612:36borkdudeE.g.:
clojure -A:depstar -m hf.depstar.uberjar MyProject.jar -v -m proj --classpath "$(clojure -Spath)"
#2020-08-1612:36borkdudeBtw, I think depstar will be able to compile with GraalVM so maybe I can even include it in babashka itself to create babashka uberjars#2020-08-1616:46Alexis VincentI’d like to distribute a deps.edn file of aliases for everyone to use in our company. But I want people to be able to have their own local ./clojure/deps.edn. Is there a way to specify a deps.edn file to merge in, either globally for all projects, or from a projects deps.edn?#2020-08-1616:58alexmillerno, not currently#2020-08-1617:29j0niIs it possible to add a list of jar files from a project sub-dir to the classpath without having to name them? I know about :local/root and I see this in the docs https://clojure.org/guides/deps_and_cli#local_jar but I have a lot of them and they aren't published anywhere. Do I just have to make up names for them and add them individually as :deps?#2020-08-1617:32alexmillerIt’s kind of a hack, but java supports on the classpath for this (literal path/ so not shell expanded). I think if you added a :paths entry for “path/*” that should work#2020-08-1708:15rickmoynihanInteresting I was wanting to do something similar too, so I could deploy an app as a zip with a directory of jars in lib but start it with the clojure command line tooling; via a :local/root; with a different deps.edn for production, rather than the one we use in dev.#2020-08-1713:43alexmillerthe other crazy thing you can do is to reference jars in your manifest (this was part of the original applet support) with the Class-Path attribute but I think that has to be explicit jars. afaik that does not support the glob stuff#2020-08-1714:55rickmoynihanyeah now you mention it I’ve seen manifests do this…. a long time since I’ve seen or written an applet tho! 😆#2020-08-1714:57alexmilleralso relevant, the manifest has a line wrapping limit and so other arcane rules so it's best to use the apis to make it rather than doing it by hand#2020-08-1714:59rickmoynihanwow I had no idea… eitherway if I do this I think it’ll be via the :local/root method, largely motivated via the new -F -X stuff#2020-08-1617:32j0nithank you @alexmiller I will try that out#2020-08-1617:59j0niamazing, thank you Alex, it works#2020-08-1619:30borkdude@seancorfield Awesome: depstar works with GraalVM and can hence be included in babashka proper so it can make its own uberjars#2020-08-1619:34borkdude$ bb -cp $(clojure -Spath) --uberjar foo.jar -m proj
Building uber jar: foo.jar
$ bb -jar foo.jar
:foo - failed: int?
#2020-08-1620:24seancorfield@borkdude Cool. As for your earlier question, depstar deliberately just includes everything on the current classpath, aside from itself.#2020-08-1620:26ghadiThat was a fundamental design decision I regret#2020-08-1620:27ghadiIn retrospect I’d use t.d.a as a library to calculate what to tar up#2020-08-1620:27ghadirather than subtracting depstar itself from the classpath#2020-08-1620:29borkdudeyep. but that can be easily changed. I've already done that in my fork#2020-08-1620:31ghadiYou using t.d.a to calculate instead?#2020-08-1620:32seancorfieldTo be honest, using the classpath has made depstar a lot easier to use for people, and a lot more predictable.#2020-08-1620:33borkdude@ghadi I just added a :classpath option that can be used to override#2020-08-1620:33borkdudehttps://github.com/borkdude/depstar/blob/d3fb96c3731d4b7987e75927e6c13497da22fae2/src/hf/depstar/uberjar.clj#L360#2020-08-1620:34seancorfieldHmm, interesting approach. Since it's easy to get clojure to produce an arbitrary classpath from deps.edn aliases etc...#2020-08-1620:34borkdudeExactly.#2020-08-1620:36ghadithis is a similar thing to what I’m trying to avoid, vectors of strings#2020-08-1620:36borkdudeSo the classpath given to babashka, produced by clojure is just passed down to depstar:
$ bb -cp $(clojure -Spath) --uberjar foo.jar -m proj
which results in a foo.jar that can be invoked with bb -jar foo.jar.#2020-08-1620:38ghadiI understand. I’m just saying the shell is not leverage compared to a clojure library. One of the things I want to do is split a list of dependencies by their rate of change for smarter distribution to containers or Lambdas#2020-08-1620:41borkdudeYou mean, make uberjars for things not changing often?#2020-08-1620:42ghadiFor containers, you don’t want every commit to result in a 60MB new uberjar, you want some structural sharing across commits#2020-08-1620:42borkdudeI guess one could use the -A or -T option for that, to make groups of deps and then push that classpath into depstar#2020-08-1620:42ghadiYeah I know you can hack it in shell, but why not program it directly in bb or clojure?#2020-08-1620:43ghadimy hope with tools.build (not that I have anything to do with it currently) is that it encourages build tasks to compose better than the lowest common denominator of the CLI#2020-08-1620:43borkdudebb will have an --uberjar option, but it can also have a bb -e '(babashka.uberjar/run {:classpath ...})-ish interface#2020-08-1620:44borkdudecan have them both#2020-08-1620:44vlaaadWait are tools.build out?#2020-08-1620:44ghadino#2020-08-1620:45seancorfielddepstar currently has zero dependencies, except Clojure itself, so it can't interfere with anyone's project dependencies -- that's worked very well in practice. I like @borkdude’s idea of just giving it an explicit classpath to build form 🙂#2020-08-1620:47vlaaadUnless I want to have the whole build happen in a single vm, and it has to have other build-specific dependencies like data.xml for updating pom.xml. I use 2 vms for that now, but I'd rather not :)#2020-08-1621:02seancorfieldI'm not sure whether you're agreeing with depstar's approach or suggesting something different?#2020-08-1620:45borkdudebb won't do AOT btw (maybe it could but I don't think it's worth it). for JVM uberjars (with AOT-ed classes), just use clojure / depstar#2020-08-1620:45borkdudebb uberjars typically will contain sources and other assets, but no classes#2020-08-1620:47borkdudeI did have depstar compile with GraalVM as is and when I didn't remove the AOT-related code, it did compile, but the binary got somewhat bloated (+ 30MB, happens when it hits a specific part of Clojure). So maybe compile just works natively, since it's just going to transform .clj into classfiles, I haven't tried it.#2020-08-1620:51borkdudeI'll try it now... just for science#2020-08-1621:15borkdudeInteresting. Compilation of a class doesn't work in a GraalVM binary because the classpath inside the binary is not set correctly for clojure RT. I guess it won't work for that reason, although technically it could maybe work.#2020-08-1621:24borkdudeGoing to try this:
(binding [*compile-path* (str tmp-c-dir)]
(clojure.lang.RT/compile (str main-class ".clj")))#2020-08-1621:25borkdudecrap, it's private ;)#2020-08-1621:38borkdudeWith a reflective hack it works. I see all kinds of AOT-ed sourced, even aot__init.class, but no aot.class. Maybe :gen-class isn't part of RT/compile? Eh.... enough for today.#2020-08-1621:42borkdudeSo the AOT part could be useful when depstar had a binary CLI version, so to speak. I don't need it for bb, but it could be useful for fast build tooling in general.#2020-08-1621:45borkdudeTo make this work clojure.lang.RT should also have a pluggable classpath resolver. It might work for Clojure files, but I wonder how it will work if code relies on .class files from the Java world. Does the Clojure compiler kind of treat .class files as data, without actually having to have them on the classpath per se, as long as you point it to the right file? I.e. no runtime need, just able to read and analyse statically?#2020-08-1622:05borkdudeI see, Compiler relies on reflection when dealing with interop, so there it will likely break, since the reflection API only works with classes that are part of the same runtime. OK, this was a nice tangent.#2020-08-1708:56borkdudeCreating uberjars with depstar native in bb now also works on Windows:#2020-08-1713:40jimberlageAnyone ever seen a message like the following after updating the :sha of a git dependency:
Error building classpath. Destination path "my-lib" already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path "my-lib" already exists and is not an empty directory
#2020-08-1713:43delaguardoyou can delete local artifacts in $HOME/.gitlibs/{libs,_repos} and restart the repl#2020-08-1713:44delaguardoI don’t know exactly what cause that problem. In my case it happens after changing :git/url#2020-08-1713:44alexmillerI believe this is the result of parallel downloading to gitlibs messing up one of the cached git dirs#2020-08-1713:50alexmillerif you clear your ~/.gitlibs (either fully or selectively) and then do clj -Sforce -Sthreads 1 that should force a re-download singlethreaded#2020-08-1713:51alexmillerwe have a ticket for this, I haven't had a chance to really look at in depth yet#2020-08-1713:55jimberlageThanks! Trying that revealed another error - technically auth-related, but testing ssh pushes/pulls to github on this repo still works for me, so I'm wondering if this is somethimg on my end or a symptom of a different bug:
Error building classpath.
#2020-08-1713:59alexmilleryou'll need to debug why it's not finding the key:
1. is ssh-agent running? if not, start it
2. does ssh-add -l list your public key? if not, add it
3. if you're on a Mac, check whether your ~/.ssh/config has Host * / IdentityFile in it, if so comment that out or remove it#2020-08-1713:59alexmillerafter that, things get trickier to debug, could be key type#2020-08-1714:05jimberlageLooks like the bit about the IdentityFile line fixed it. Thank you#2020-08-1714:14alexmiller👍#2020-08-1911:10borkdudeWhy does clojure not warn about an unexisting :test alias?
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "6f0d001d040b1a0b0a2f222d3f5d5f5e56"}, :content ("[email protected]")}
#2020-08-1911:11borkdudeAlso, I figured it would be useful to have something like: clojure -CR:test to get the same as clojure -C:test -R:test: use case, I want to run a REPL with the same paths and deps as the :test alias, just not execute the main#2020-08-1913:11alexmilleryou can use clojure -A:test to do that#2020-08-1913:12alexmilleroh, "and not execute the main" - well for that you need the -C / -R#2020-08-1911:28vlaaadI think :test exists in install deps?#2020-08-1911:58dharriganindeed, in my install on Arch, in /usr/share/clojure/deps.edn it has test as an alias#2020-08-1912:56souenzzoThis file
https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/deps.edn
The "global" deps#2020-08-1916:59seancorfield@borkdude https://github.com/seancorfield/depstar/issues/34#2020-08-1917:30alexmiller-Scp is an existing clj option btw#2020-08-1917:40borkdudeThat's useful, but it only affects the runtime classpath. This feature is about overriding the classpath from which the uberjar is created, which makes it more flexible. The classpath can be generated from other tools like leiningen, boot, clojure, etc, whatever source.#2020-08-1917:46alexmillergotcha, might be something good to add to tools.build's uber stuff then :)#2020-08-1919:41Janne Sauvala@alexmiller I keep hearing references to tools.build (since your Inside Clojure -post). Is there more detailed description of it somewhere already or can we expect something coming out soonish? 🙂#2020-08-1920:01seancorfieldI suspect the answers are "no" and "maybe" -- depending on your definition of "soon" 🙂#2020-08-1920:09alexmiller^^#2020-08-1920:09alexmillertrying to pull together a lot of strands right now, not sure on timing yet#2020-08-2005:59Janne SauvalaGotcha. I’m exited to hear tools.build could be a thing. I really enjoy working with tools.deps and the latest features look great 👍:skin-tone-2:#2020-08-1917:03seancorfield@alexmiller https://github.com/seancorfield/depstar/issues/35 -- good idea you posted in #beginners !#2020-08-1917:30alexmillerwe would definitely like to encourage more use of -X - considering a bunch of additional features that may continue to make it more attractive for tools#2020-08-1917:52vlaaadFeatures like...? :)#2020-08-1917:59alexmillerwell, still designing :)#2020-08-1918:01alexmillerbut really trying to simplify invocation in several ways#2020-08-1918:39seancorfieldUpdating hf.depstar.uberjar:
(defn run
"Generic entry point for all invocations.
Can be used with `clojure -X`:
In `:aliases`:
:depstar {:extra-deps {seancorfield/depstar {:mvn/version ...}}}
:uberjar {:fn hf.depstar.uberjar/run
:args {:aot true :jar-type :uber}}
Then run:
clojure -R:depstar -X:uberjar :jar MyProject.jar :main-class project.core
If the destination JAR file and main class are fixed, then could be
added to `:args` in `deps.edn`:
:depstar {:extra-deps {seancorfield/depstar {:mvn/version ...}}}
:uberjar {:fn hf.depstar.uberjar/run
:args {:aot true :jar MyProject.jar
:jar-type :uber :main-class project.core}}
Both `:jar` and `:main-class` can be specified as symbols or strings."
It would be nice if -X acted like -R .. -X .. so you didn't have to also specify -R:depstar here and could have the :extra-deps in the :uberjar alias.#2020-08-1918:41alexmillerwhat I'm working on right now is in that area#2020-08-1918:42alexmillerall over it!#2020-08-1917:30alexmillerand easier to use#2020-08-1917:42borkdude@seancorfield One other thing that came up in #beginners: the non-standard way of providing the name of the uberjar. It would be easier to compose if that could be done using a named parameter like --uberjar foo.jar#2020-08-1917:43borkdudeSee https://clojurians.slack.com/archives/C053AK3F9/p1597841581095000#2020-08-1918:40seancorfield@borkdude Yeah, I'm making it :jar for the -X option and it will also be -J / --jar on the command-line (as well as just accepting the filename anywhere).#2020-08-1918:47dakraNice :)#2020-08-1918:41seancorfieldSo @daniel415’s example will work exactly as he would like.#2020-08-1918:41borkdude👍#2020-08-1920:17seancorfieldOK, in light of earlier discussions, the develop branch of depstar has several new features requested this morning: support for clojure -X, support for overriding the classpath used for building the JAR, and reworked argument parsing so the JAR filename can be specified "anywhere" (before/after/in the middle of the options). Please take it for a spin and report back any issues (either on GitHub or here). seancorfield/depstar {:git/url "" :sha "c461ba830e069edf0f011169c18595a4a8f917fc"} Thanks in advance#2020-08-2012:41timo👋
How do I pass a version to clj -Spom?#2020-08-2012:43timoI need it for CI. or is there a more clever way to update the version in the pom.xml for clojars-deployment?#2020-08-2012:48delaguardoI’m using sed 🙂
https://github.com/xapix-io/axel-f/blob/master/.github/workflows/publish.yml#L45-L46#2020-08-2013:15timosure, this is another option#2020-08-2013:15timothanks#2020-08-2012:48dominicm@timok generally you cannot. clj -Spom only updates your deps (and generates a basic outline).#2020-08-2012:48dominicmThere's garamond which can do that though#2020-08-2012:50jose@timo you can use the maven versions plugin: mvn versions:set versions:set-scm-tag -DnewVersion="${version}"#2020-08-2012:51timoah thanks @dominicm @jlle. then I will keep the pom.xml in the repo.#2020-08-2012:52joseif you want to publish from github, I wrote a github action: https://github.com/jlesquembre/clojars-publish-action#2020-08-2013:09timothanks @jlle but it's actually circleci but I will have a look 👍#2020-08-2014:41joseDo you think that the new -T CLI option can be a better alternative to user.clj? I can imagine moving some repl helpers I have in my user.clj to a function and invoke that function explicitly with -T#2020-08-2014:47alexmillerno, that doesn't seem right#2020-08-2014:48alexmillerif you're starting a repl, you can use the clojure.main -i to do things like that#2020-08-2014:48alexmiller-T is for invoking external tools (although I think -T is actually going to go away anyways)#2020-08-2016:06seancorfieldWas -T intended to be a specialized -R/`-C` combination?#2020-08-2016:07seancorfieldI was looking on the CLI reference page for where you can use :deps in an alias and that seems to just be -T -- but I thought it was valid in some other use case...?#2020-08-2016:07borkdudeI think -T did not include the deps on the top level. Which I can see being useful. Kind of like a system wide alias, but for projects#2020-08-2016:10seancorfieldIt feels "simpler" to just allow :deps / :paths in any alias with the expectation that it overrides the top-level values... but I guess there's always the question of what happens if you specify two aliases and the first has :deps / :paths to override and then the second alias has :extra-deps / :extra-paths to add to those? Will people expect that to work on will :deps / :paths always "win"?#2020-08-2016:10alexmiller:deps/paths replacement was previously only available via -A, was pulled out as -T in past#2020-08-2016:10alexmillerwe are reworking all of this stuff right now#2020-08-2016:10seancorfieldToo many unanswered questions 🙂 Needs more Hammock Time!#2020-08-2016:10alexmilleralready did it#2020-08-2016:10borkdudepaths replacement = classpath overrides?#2020-08-2016:11alexmillerno#2020-08-2016:11alexmiller:paths replaces the :paths in your project deps.edn#2020-08-2016:11borkduderight#2020-08-2016:11alexmillerwhich is admittedly a very unusual use case#2020-08-2016:11borkdudecan be useful for a project within a project?#2020-08-2016:11alexmillerwe're going to rename these to :replace-deps and :replace-paths#2020-08-2016:12borkdudeyeah, that's clearer#2020-08-2016:12seancorfieldAnd if there are multiple of those, it'll be like :main-opts and only one gets picked?#2020-08-2016:12alexmillerit's not worth me going through all this by piece - it's all getting updated#2020-08-2016:13alexmillerso let me go work on that :)#2020-08-2016:13borkdudean analogy with :classpath-overrides {foo/bar nil} could be :remove-deps and :remove-paths#2020-08-2016:15seancorfieldFWIW, I'm liking the -X stuff (modulo always wanting :extra-deps to be respected by it) and I'm thinking it will simplify several pieces of tooling at work, since we can remove multiple namespaces containing just a -main function that exists solely to parse/route arguments to other functions that take a hash map 🙂#2020-08-2016:16alexmillerthat's the idea (and usage stuff will be improved)#2020-08-2016:16seancorfield(our current problem is a couple of our servers are too old to be able to run the install script so I have to update them manually -- but that's not Clojure's problem 🙂 )#2020-08-2016:16alexmilleralso working on ideas to make ad hoc -F variant a bit more useful for the case of one namespace with many entry point fns#2020-08-2016:17alexmillerhave I talked about -F yet? I can't even remember now#2020-08-2016:17borkdudeno, go on ;)#2020-08-2016:17seancorfield-F is like -X only without needing an alias 🙂#2020-08-2016:18alexmilleryeah, invoke ad hoc function#2020-08-2016:18borkdudewhy even have X then?#2020-08-2016:18alexmillerlets you package default args and not repeat the fn at cli#2020-08-2016:18alexmillerboth useful in different contexts#2020-08-2016:18seancorfieldInvoking via an alias is more flexible. Yeah, default args, simpler name, will be even more useful when it respects :extra-deps 🙂#2020-08-2016:18borkdudeyou can stuff default args in the function itself though?#2020-08-2016:19seancorfieldSure, but you could have different aliases with different default args. That's a use case that we could leverage at work.#2020-08-2016:19borkdude-A:foo -F foo/bar, respects extra deps of foo alias?#2020-08-2016:20seancorfieldCurrently, you'd need -R:foo -X:foo for a function alias -- repetitive! -- so if -X respects :extra-deps you could just do -X:foo#2020-08-2016:22alexmillerthis is coming#2020-08-2016:23seancorfieldYeah, at this point I'm just sort of assuming it will do that 🙂#2020-08-2016:21seancorfieldBut -F is definitely useful for ad hoc command-line work.#2020-08-2016:22borkdudeinstead of creating different aliases with different defaults, one could also just write multiple functions in one namespace. I don't directly see the benefit of one over the other, -A:foo -F foo/foo, -A:bar -F foo/bar.#2020-08-2016:22alexmillersure if you bake it into the code#2020-08-2016:23alexmillerputting it in deps.edn is default but configurable#2020-08-2016:23borkdudeI guess having it closer together is one benefit#2020-08-2017:04dominicm-F seems like -e? But with less (#2020-08-2017:07alexmiller-e is arbitrary expression, -F is "invoke a functional entry point that takes a map" so it's much more specific#2020-08-2101:59steveb8nQ: Is this a silly idea? I want a combination of git deps and “-LATEST” behviour for my CI/test builds. For prod builds I want static git shas but for automated/test builds, it would be great to automatically use the most recent git sha. Is this even possible?#2020-08-2102:35souenzzoTemporal-basead versions are always bad IMHO
If your CI fail and you run it again, it may take a new commit and success again. So success and fail says nothing.
Pass the commit ID to your CI or fetch and use a sed / template.
Then in case of success/fail, you will know which code succeed/failed#2020-08-2102:35souenzzoAnd you will be able to run again the same CI in a older commit/retry#2020-08-2102:42steveb8nthx. fetch and sed is an interesting idea. I could even use bb for this script.#2020-08-2102:43steveb8nhow would the script apply the latest sha? by writing it into the deps.edn? or is there some other trick?#2020-08-2102:45steveb8nfyi: the reason I want this is to avoid the endless commits I currently do for all my git deps during dev/test. but I really like using git deps in CI#2020-08-2102:55seancorfieldRunning CI and production against different versions seems like a recipe for hard-to-debug disaster...#2020-08-2102:56seancorfieldI can understand wanting your local dev-only tooling to always pick up the RELEASE (or perhaps occasionally the LATEST) release but I really don't want any variance in versions between dev, CI, production for anything I depend on.#2020-08-2102:57seancorfield(so I guess my answer to your actual Q would be "yes, it's a silly idea" IMO)#2020-08-2103:19steveb8nthat is useful perspective. I’ll apply a hammock to this a little more. thanks#2020-08-2103:24steveb8nI should have mentioned that these deps are all projects that I control but have split apart for re-use and/or separation of concerns. so I can be sure I know when changes are being introduced. I would never do this for an external dependency#2020-08-2103:34seancorfieldConsider using :local/root maybe, if you have them all checked out locally?#2020-08-2103:34seancorfieldYou can always use a :dev alias to use :override-deps to replace the regular deps with your local ones.#2020-08-2103:43steveb8nagreed. I am using :local/root for these deps for localhost. the annoying commits are required when I push a lib which I depend on in N other projects and then have to update shas in all of them#2020-08-2103:44steveb8nideally CI/test would use latest for all these deps. then when I cut a prod release, I would update the shas (once) for all the deps#2020-08-2103:45steveb8nbut I can see that I could introduce human error later i.e. forget 1 sha update in the prod build. that would be bad#2020-08-2103:45steveb8nthat would qualify as “silly”#2020-08-2103:46steveb8nthis maintenance problem seems to be pushing me towards a mono-repo but I like keeping these things separate#2020-08-2104:07seancorfieldOur 100K+ line codebase at work is in a monorepo, with 30+ subprojects, and we build about a dozen deployment artifacts from that...#2020-08-2104:14steveb8nit’s something I will think about. thanks#2020-08-2106:39borkdude@U0510KXTU I’m using git submodules to prevent endless commits updating git deps. Easy to update them all to latest using one git command. Works with all build tools.#2020-08-2107:25steveb8nthanks. I’ll look into that as well. I’ve never used them before but, if they add no complexity, I’ll give them a try#2020-08-2107:26borkdude@U0510KXTU I'm using them in babashka, I have 4 of them now.#2020-08-2107:27steveb8ncool. I’ll look at your repos when I dig deeper#2020-08-2107:27borkdudeNot everyone likes them, I'm not entirely sure why, they work well for me.#2020-08-2111:29practicalliI would certainly keep specific library dependency versions the same for any CI pipeline that had any chance of pushing things into production.
Are you able to set up separate CI pipeline that uses a different alias to pull 'latest' versions of libraries?
I prefer using :local/root rather than mono-repo or sub-modules as they feel simpler for the way I work. Nothing wrong with those approaches though...#2020-08-2111:31borkdudelocal/root works, but not if the repo you're including is not part of the codebase proper, unless you're writing a script to set up the filesystem in CI the exact same way as on your machine#2020-08-2111:34practicalliIs that the case when using jars built from other projects, I thought that worked. Maybe I am mis-remembering.#2020-08-2111:35borkdudeI mean, with local/root to some other project, you'll also have to git clone that other project in CI right#2020-08-2111:36practicalliAh, yes.#2020-08-2123:22steveb8nI feel the same @U05254DQM in that I like them all separate. it has the added benefit of allowing me to outsource work on projects without needing to grant access to all#2020-08-2123:23steveb8nso for me it’s.. local deps using local/root, CI using git/sha and prod using the same git/sha#2020-08-2123:24steveb8nand the maintenance annoyance is updating the git shas for the CI builds anytime upstream projects change.#2020-08-2123:25steveb8nfrom this thread, I’ve realised that this is also a good thing because it means that my CI builds always match prod builds. thus it’s probably the lesser of all evils, despite the chattiness of commits#2020-08-2408:15rickmoynihan@U04V15CAJ: I’ve done the git submodule thing many times in the past. I’ve found it works quite well if you’re careful and have infrequently changing dependencies, but it can become pretty painful if you change branches a lot and have fast moving dependencies.
The detached heads in the submodules can also lead to people unwittingly making many kinds of mistakes.#2020-08-2408:20borkdudegit pull --recurse-submodules#2020-08-2408:21borkdudeI almost never have such mistakes, but maybe for contributors it's different?#2020-08-2108:37borkdudeWondering, is tools.build also going to have a test runner like https://github.com/cognitect-labs/test-runner or is this a separate concern?#2020-08-2112:29alexmillerNot building any new test runner but you can make tasks that do whatever#2020-08-2110:34otfromHi, I'm trying to reference the head of the manual-connector-anchors branch on dali here: https://github.com/stathissideris/dali/tree/manual-connector-anchors#2020-08-2110:34otfromand I'm struggling a bit with the deps.edn. Does anyone have a good example of how to reference a branch on github at a :git/url ?#2020-08-2112:32alexmillerGit deps always use shas so you don’t need to specify the branch#2020-08-2112:32alexmillerSha is sufficient#2020-08-2115:03otfromthx! I've got it working now. It was a problem with the deps.edn in the repo I was referencing.#2020-08-2115:03alexmillerah#2020-08-2110:41otfromI have it working locally in the mean time#2020-08-2111:20otfromfixed: https://github.com/stathissideris/dali/pull/20#2020-08-2422:55tolitiusneed to build a thin jar with one namespace aot compiled. the namespace has no dependencies:
{:paths ["src" "classes"]
:deps {}
:aliases {:jar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.97"}}
:main-opts ["-m" "hf.depstar.jar" "yang.jar"]}}}
$ clj -e "(compile 'yang.java)"; clj -A:jar
it does work, but it also brings these along:
0 Mon Aug 24 18:45:40 EDT 2020 clojure/core/
0 Mon Aug 24 18:45:40 EDT 2020 clojure/core/specs/
932 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__40.class
1178 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__65$fn__71.class
798 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__46.class
929 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__27.class
936 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__101.class
929 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__25.class
...
I might be missing some args in depstar?#2020-08-2422:58seancorfield@tolitius Clojure core uses specs for language features (as macros), so those get dragged in because of that.#2020-08-2422:59tolitiusthank you. is there a way to exclude these?#2020-08-2422:59tolitiuswhen assembling a jar maybe#2020-08-2422:59seancorfieldYou'd have to manually remove them after your compile step right now.#2020-08-2423:00seancorfieldFeel free to open an issue on GH with those repro steps and I'll see about adding an option to exclude stuff via regex.#2020-08-2423:01tolitiusgreat, would that be a depstar repo?#2020-08-2423:01seancorfieldseancorfield/depstar#2020-08-2423:01tolitiusyep, will do. thank you Sean#2020-08-2423:02seancorfieldThere's already quite a few changes on develop so if I incorporate that exclusion this week, I'll cut a 1.1 release.#2020-08-2423:03tolitiusoh.. nice. that’s fast#2020-08-2423:07seancorfieldI'm off work Thu/Fri so I expect I'll do a round of OSS updates and releases before next Monday 🙂#2020-08-2423:08tolitiushttps://github.com/seancorfield/depstar/issues/37
wut it’s going to take a week?? )
yea, of course this is not urgent. I can remove these manually for now. thanks again#2020-08-2423:25tolitiusin case someone sees this before the next monday:
clj -e "(compile 'yang.java)"; clj -A:jar; zip -d yang.jar clojure/\*;
does it#2020-08-2423:53seancorfield@tolitius If you don't mind depending on develop, you can test that right now.#2020-08-2423:59seancorfieldNote that it excludes files not directories, so you'd still get the (empty) clojure/core/ and clojure/core/specs/ folders if you --exclude "clojure/.*"#2020-08-2500:24tolitius:main-opts ["-m" "hf.depstar.jar" "yang.jar" "--exclude" "clojure/core/specs/alpha.*"]}
756 Mon Aug 24 20:21:36 EDT 2020 yang/java$map__GT_edn$iter__130__134.class
3064 Mon Aug 24 20:21:36 EDT 2020 yang/java__init.class
1316 Mon Aug 24 20:21:36 EDT 2020 yang/java$fn__128.class
2003 Mon Aug 24 20:21:36 EDT 2020 yang/java$map__GT_edn$iter__130__134$fn__135$fn__136.class
0 Mon Aug 24 20:23:37 EDT 2020 clojure/
0 Mon Aug 24 20:23:37 EDT 2020 clojure/core/
0 Mon Aug 24 20:23:37 EDT 2020 clojure/core/specs/
0 Mon Aug 24 20:23:37 EDT 2020 tolitius/
1867 Mon Aug 24 20:21:36 EDT 2020 tolitius/Yang.class
yep, it works )#2020-08-2500:32seancorfieldCool, thanks for testing!#2020-08-2511:47Janne SauvalaCan I specify version ranges in deps.edn for dependencies?#2020-08-2512:08vlaaadno#2020-08-2517:04seancorfield@janne.sauvala Yes, you can, but you shouldn't. The string used for the version in :mvn/version is just passed through to the underlying library, e.g.,
(! 1176)-> clj -Sdeps '{:deps {hiccup {:mvn/version "(,1)"}}}'
DEPRECATED: Libs must be qualified, change hiccup => hiccup/hiccup
Downloading: hiccup/hiccup/1.0.0-SNAPSHOT/maven-metadata.xml from clojars
Downloading: hiccup/hiccup/1.0.0-SNAPSHOT/hiccup-1.0.0-20120218.070543-1.pom from clojars
Downloading: hiccup/hiccup/1.0.0-SNAPSHOT/hiccup-1.0.0-20120218.070543-1.jar from clojars
Clojure 1.10.1
user=> #2020-08-2517:06seancorfieldCurrent version is 1.0.5 but you can get the highest version below that
(! 1177)-> clj -Sdeps '{:deps {hiccup {:mvn/version "(,1.0.5)"}}}'
DEPRECATED: Libs must be qualified, change hiccup => hiccup/hiccup
Downloading: hiccup/hiccup/maven-metadata.xml from clojars
Downloading: hiccup/hiccup/1.0.4/hiccup-1.0.4.pom from clojars
Downloading: hiccup/hiccup/1.0.4/hiccup-1.0.4.jar from clojars
Clojure 1.10.1
user=>
But version ranges are "Bad(tm)" and should be avoided.#2020-08-2517:06seancorfield^ @vlaaad FYI#2020-08-2517:15alexmillerit's not actually passed through, it's resolved in canonicalization to the max in range#2020-08-2517:16alexmillerbut a) version ranges have many known weirdnesses and b) as a non-specific version, they also break clj's caching (you will retain the same cached classpath even if newer versions become available)#2020-08-2517:16Janne SauvalaInteresting to see it works, thanks @seancorfield! I realized I haven’t seen anyone using version ranges so I was wondering were those intentionally not supported#2020-08-2517:16alexmillerthey are officially not supported#2020-08-2517:16alexmillerbut unofficially handled because they exist in the wild, particularly with transitive deps#2020-08-2517:18seancorfield@alexmiller Good to know re: canonicalization -- I passed in a bad range and the exception came from Aether so I assumed it was just being passed through. My bad:
Unbounded version range (1,
org.eclipse.aether.resolution.VersionRangeResolutionException: Failed to resolve version range for hiccup:hiccup:jar:(1,: Unbounded version range (1,
at org.apache.maven.repository.internal.DefaultVersionRangeResolver.resolveVersionRange(DefaultVersionRangeResolver.java:137)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveVersionRange(DefaultRepositorySystem.java:247)
#2020-08-2517:19seancorfield> I haven’t seen anyone using version ranges
Because they're bad and folks avoid them 🙂#2020-08-2517:24Janne SauvalaYes. Too bad I have seen them used quite a lot in npm and python world. It is good that bad practice hasn’t polluted clojure community 🙂#2020-08-2517:19alexmillerwell the resolution happens via aether#2020-08-2517:20alexmillerI think if you had more of stack, you'd probably see canonicalize in there somewhere#2020-08-2621:26jhacksTrying out antq (https://github.com/liquidz/antq) and depot (https://github.com/Olical/depot) with these one-liners I get many hundreds of lines of DEBUG statements:
- clojure -Sdeps '{:deps {antq/antq {:mvn/version "RELEASE"}}}' -m antq.core
- clojure -Sdeps '{:deps {olical/depot {:mvn/version "2.0.1"}}}' -m depot.outdated.main
Is this normal or is there something up with my system?#2020-08-2621:30seancorfield@jhacks What do you mean by "hundreds of lines of DEBUG statements"?#2020-08-2621:31seancorfieldI just tried antq.core on a project and got this:
> clojure -Sdeps '{:deps {antq/antq {:mvn/version "RELEASE"}}}' -m antq.core
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
| :file | :name | :version | :latest-version |
|----------+-----------------------------------------------------------------+------------+-----------------|
| deps.edn | com.cognitect/test-runner | | Failed to fetch |
| | com.h2database/h2 | 1.4.199 | 1.4.200 |
| | com.microsoft.sqlserver/mssql-jdbc | 8.2.1.jre8 | 8.4.0.jre14 |
| | com.zaxxer/HikariCP | 3.4.2 | 3.4.5 |
| | io.zonky.test.postgres/embedded-postgres-binaries-darwin-amd64 | 12.2.0 | 12.3.0 |
...
which seems perfectly reasonable.#2020-08-2621:32seancorfielddepot produced this
> clojure -Sdeps '{:deps {olical/depot {:mvn/version "2.0.1"}}}' -m depot.outdated.main
Downloading: olical/depot/2.0.1/depot-2.0.1.pom from clojars
Downloading: olical/depot/2.0.1/depot-2.0.1.jar from clojars
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Checking for old versions in: deps.edn
org.clojure/java.data {:mvn/version "1.0.78"} -> {:mvn/version "1.0.86"}#2020-08-2621:36jhacks@seancorfield For this command clojure -Sdeps '{:deps {olical/depot {:mvn/version "2.0.1"}}}' -m depot.outdated.main
I get many lines like this before ending with the usual output:
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << Age: 7291592
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Served-By: cache-bwi5137-BWI, cache-wdc5525-WDC
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Cache: HIT, HIT
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Cache-Hits: 1, 1
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Timer: S1598477735.173106,VS0,VE1
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.client.DefaultHttpClient - Connection can be kept alive indefinitely
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<html>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<head><title>404 Not Found</title></head>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<body bgcolor="white">[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<center><h1>404 Not Found</h1></center>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<hr><center>nginx</center>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "</body>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "</html>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection [id: 49][route: {s}->] can be kept alive indefinitely
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection released: [id: 49][route: {s}->][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 100]
17:35:35.195 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:35:35.196 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51708<->199.232.64.209:443 closed
17:35:35.196 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51708<->199.232.64.209:443 closed
17:35:35.197 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:35:35.198 [clojure-agent-send-off-pool-2] DEBUG org.eclipse.aether.internal.impl.TrackingFileManager - Writing tracking file /Users/joe/.m2/repository/selmer/selmer/resolver-status.properties
Checking for old versions in: deps.edn
cljs-ajax {:mvn/version "0.8.0"} -> {:mvn/version "0.8.1"}
com.google.javascript/closure-compiler-unshaded {:mvn/version "v20200504"} -> {:mvn/version "v20200719"}
day8.re-frame/http-fx {:mvn/version "0.1.6"} -> {:mvn/version "0.2.1"}
metosin/jsonista {:mvn/version "0.2.6"} -> {:mvn/version "0.2.7"}
org.clojure/core.async {:mvn/version "1.2.603"} -> {:mvn/version "1.3.610"}
org.postgresql/postgresql {:mvn/version "42.2.14"} -> {:mvn/version "42.2.16"}
re-frame {:mvn/version "1.0.0"} -> {:mvn/version "1.1.1"}
thheller/shadow-cljs {:mvn/version "2.10.17"} -> {:mvn/version "2.11.0"}
#2020-08-2621:36jhacksSorry about the formatting#2020-08-2621:40jhacksFor clojure -Sdeps '{:deps {antq/antq {:mvn/version "RELEASE"}}}' -m antq.core
I get lines like this:
17:38:47.822 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:38:47.822 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51766<->151.101.193.128:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51766<->151.101.193.128:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51765<->199.232.64.209:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51765<->199.232.64.209:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51769<->199.232.64.209:443 closed
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51769<->199.232.64.209:443 closed
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
| :file | :name | :version | :latest-version |
|----------+-------------------------------------------------+-----------+-----------------|
| deps.edn | cider/piggieback | 0.5.0 | 0.5.1 |
| | cljs-ajax/cljs-ajax | 0.8.0 | 0.8.1 |
| | com.cognitect/test-runner | | Failed to fetch |
| | com.google.javascript/closure-compiler-unshaded | v20200504 | v20200719 |
| | day8.re-frame/http-fx | 0.1.6 | 0.2.1 |
| | metosin/jsonista | 0.2.6 | 0.2.7 |
| | org.clojure/core.async | 1.2.603 | 1.3.610 |
| | org.postgresql/postgresql | 42.2.14 | 42.2.16 |
| | re-frame/re-frame | 1.0.0 | 1.1.1 |
| | thheller/shadow-cljs | 2.10.17 | 2.11.0 |
#2020-08-2621:47jhacksI’m using clojure from homebrew and just updated it to see if that was causing it, but no change. There isn’t a debug setting somewhere with deps.edn?#2020-08-2621:51jhacksI tried it on another project that uses deps.edn (I only have two) and I don’t get the DEBUG output. It must be something specific to the project. Thanks for trying out the commands on your side @seancorfield!#2020-08-2621:54seancorfieldSomething in your project is enabling logging (at debug level) and those DEBUG messages are coming from the machinery that fetches metadata about artifact versions.#2020-08-2621:55seancorfieldYou may be able to resolve that by adding org.slf4j/slf4j-nop {:mvn/version "1.7.25"} to your command-line deps when running antq and/or depot @jhacks#2020-08-2621:59jhacks@seancorfield Wow! That got rid of the DEBUG statements on both of the commands. Thanks so much, that’s good to know about.#2020-08-2622:00seancorfieldI use it with clj-new to suppress logging in project templates as it is generating them 🙂#2020-08-2622:05jhacksThanks for this video, btw: https://www.youtube.com/watch?v=CWjUccpFvrg I found it very helpful. I’ve been using lein and just getting started with deps.edn. clj-new is on my list to check out. 🙂#2020-08-2622:14practicallihttps://practicalli.github.io/clojure/clojure-tools/using-clojure-tools.html covers using Clojure CLI tools and several community tools.
Sean and I both have examples of ~/.clojure/deps.edn configurations you may find of use
https://github.com/practicalli/clojure-deps-edn#2020-08-2622:20jhacksThanks @U05254DQM. I found this tip extremely helpful: https://practicalli.github.io/clojure/clojure-tools/deps-edn-aliases.html#only-one-main-namespace#2020-08-2622:21jhacksA lot of good info there 🙂#2020-08-2622:23seancorfieldThe example of mine that John refers to above is at https://github.com/seancorfield/dot-clojure#2020-08-2622:25practicalliI learnt most things from Sean (and Alex too)#2020-08-2812:10rickmoynihanAre there any plans to support data-readers in deps.edn :args data?#2020-08-2812:14rickmoynihanThe use case is defining overrides for an integrant system, allowing #ig/ref.
Obviously it’s also easy to put that system in a separate file, and read that, so not a big deal, and I can imagine that it’d add complexity to tools.deps. Just curious.#2020-08-2812:43alexmillerI don’t think there’s any need for that#2020-08-2812:44alexmillerOr do you mean for data in aliases in deps.edn?#2020-08-2812:48dominicmfor context #ig/ref creates a record type. I suspect Rick is trying to pass in the integrant system to run via deps.edn aliases.#2020-08-2813:04alexmillerwhen you use -X the alias data is actually loaded in the context of the program you're running, so the data readers available via data_readers.clj[c] are available (but not currently being used in the edn/read, so seems like this shouldn't be too hard#2020-08-2813:41rickmoynihanYeah it’s for data in aliases#2020-08-2813:43rickmoynihanAnd yeah I was thinking the same, using data_readers.clj(c) would be great for this.#2020-08-2813:45alexmillerI will log a jira for this, seems a reasonable thing to want#2020-08-2813:46rickmoynihan@dominicm’s suspicions are right… Essentially I’m writing a script, that wants to leverage some integrant components pre-defined in libraries. I currently have the system defined in a var in my program, and it would be useful to be able to override some of the components configs via args.
I could obviously just put it in a separate edn config file; but it’s another config file (sigh) for something that’s intended to be pretty minimal.#2020-08-2813:47alexmilleryep, got it#2020-08-2813:48rickmoynihanIt would be fab if tools.deps could do this… it’d make configuring and overriding bits of integrant systems much nicer.#2020-08-2813:49rickmoynihanThough I warn you, it’ll be a slippery slope, I’ll probably next want data aliases and command line args to be meta-merged 😆#2020-08-2813:53alexmillerhttps://clojure.atlassian.net/browse/TDEPS-163#2020-08-2813:53alexmillerwhat does "meta-merged" mean?#2020-08-2814:16rickmoynihanWell I was just joking about supporting meta-merge 🙂
But I meant:
https://github.com/weavejester/meta-merge
which is very useful for doing deep merges of data.
We tend to structure our integrant systems into profiles that get applied and meta-merged in a certain order.#2020-08-2814:20rickmoynihanSo in a tools.deps meta-merge context you could have data aliases for each of those profiles, and meta-merge them all together in the specified order, with -Sdeps and command line args on top etc.
I was only joking though assoc-in makes more sense for tools.deps.#2020-08-2814:20alexmillerwell, that's there for -X with an :args map and pathed data on the command line#2020-08-2814:21alexmillerI'm not going to use meta-merge on deps data, but your program can combine data from multiple aliases that you load in any way you want#2020-08-2814:21alexmillerby loading it via the runtime basis#2020-08-2814:22rickmoynihan👍 Ahh yes that’s true — I’d forgotten I could do that.#2020-08-2814:23rickmoynihanI think that would work quite nicely actually 🤔#2020-08-2814:24alexmillerhttps://clojure.org/reference/deps_and_cli#_basis_injection#2020-08-2814:24rickmoynihanI already have it slurped in my REPL 😁#2020-08-2814:24rickmoynihanThanks#2020-08-2814:25alexmillerI guess that code could use data readers too :)#2020-08-2814:25rickmoynihanYeah was just going to ask about that#2020-08-2814:25rickmoynihan👍#2020-08-2814:26alexmilleredn/read-string takes an optional opts map where you can pass :readers#2020-08-2814:27alexmiller(def basis (edn/read-string {:readers *data-readers*} (-> (System/getProperty "clojure.basis") jio/file slurp)))
#2020-08-2814:27alexmillersomething like that?#2020-08-2814:28rickmoynihanyeah that would work nicely with TDEPS-163#2020-08-2814:28alexmillerthere are other places that edn/read-string that you don't control, so those should do this too#2020-08-2814:29alexmillerinside tools.deps (deps reader) and the exec stub (command line)#2020-08-2814:29rickmoynihanYeah I can’t do anything without TDEPS-163 also supporting the readers properly#2020-08-2814:30alexmillerI probably won't get to it today, but it shouldn't be a big deal#2020-08-2814:32rickmoynihanNo rush, but it’ll be great to see this#2020-08-2814:33rickmoynihanOther use cases for us are that we like to represent URIs as data, not strings… and we have quite a lot of URIs in config data.#2020-08-2813:55alexmillerI'm guessing you mean deps.edn and -Sdeps alias data merged?#2020-08-2813:57alexmillerthose will be merged now, but perhaps not in the way you want :)#2020-09-0120:11ghadi@seancorfield do you use anything to deploy artifacts to S3 private repos with deps.edn?#2020-09-0120:11ghadianyone can answer, just tagging Sean because he has the biggest dotfile#2020-09-0120:12seancorfieldNope, sorry. We don't use S3 for repos.#2020-09-0120:30kenny@ghadi I wrote this to do a subset of the stuff required https://github.com/kennyjwilli/s3-mvn-upload.#2020-09-0120:30ghadithanks#2020-09-0207:39rickmoynihanHmm I should really think about open sourcing our s3 deployer#2020-09-0306:29vlaaaddeps-deploy is so simple, I'm almost ready to just use aether#2020-09-0306:30vlaaadI mean this: https://github.com/slipset/deps-deploy/blob/master/src/deps_deploy/deps_deploy.clj#2020-09-0306:31vlaaadwhat's missing is an easy way to build pom.xml, currently a lot of build tooling complexity is either about updating or reading the pom#2020-09-0306:31vlaaadI hope tools-build has something in this area#2020-09-0306:33vlaaadI tailored deps-deploy to my needs (e.g. use args instead of env for clojars username/token) and it's almsot gone: https://github.com/vlaaad/reveal/blob/master/build/deploy.clj#2020-09-0307:36rickmoynihanYeah our one (not yet opensourced) is actually almost identical to deps deploy (it just wraps pomegrante/aether too), except it also supports private s3 wagons. I’m almost tempted to just patch deps-deploy with the code to do that, rather than extract and open ours.#2020-09-0307:51dominicmI'd like to see thought go into credential management fwiw. I had a stab and discovered it's very tricky.#2020-09-0308:35rickmoynihanThe other option is I guess to use cognitects s3 api to drive it; I think that hooks into all the possible AWS credential mechanisms#2020-09-0308:36rickmoynihanThough I think that would require you facading things for aether/pomegranate#2020-09-0308:40dominicmThat's fine for s3, less so for clojars creds#2020-09-0414:51borkdudeI'm thinking about a feature for clj-kondo where library authors can publish config specific to their library and clj-kondo can detect this somehow. So their artifacts should have some predictable and unique location for this config. Has something like this been done for other tools, so I could look at some prior work?#2020-09-0414:53alexmillerI've been thinking about exactly this lately for clj, but have not settled on an answer#2020-09-0414:54alexmillerone answer in the new clj world is to put it in :aliases of course#2020-09-0414:54alexmillerin general, we're trying to avoid "magic name" though#2020-09-0414:54borkdudeyup but deps.edn itself isn't often part of a published artifact right#2020-09-0414:55alexmillerright - have been thinking about some way to allow tools to be self-describing#2020-09-0415:11borkdudeI guess it doesn't have to be unique. If every library would publish their exported config under /clj-kondo, I guess you could just do .getResources. But maybe that doesn't work for dirs.
user=> (enumeration-seq (.getResources (.getContextClassLoader (Thread/currentThread)) "README.md"))
(#object[java.net.URL 0x50acf55d "jar:file:/Users/borkdude/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20170809-b9c14c6b/google-closure-library-third-party-0.0-20170809-b9c14c6b.jar!/README.md"] #object[java.net.URL 0x3cae7b8b "jar:file:/Users/borkdude/.m2/repository/org/clojure/google-closure-library/0.0-20170809-b9c14c6b/google-closure-library-0.0-20170809-b9c14c6b.jar!/README.md"])
#2020-09-0415:13borkdudeI guess it does work:
user=> (enumeration-seq (.getResources (.getContextClassLoader (Thread/currentThread)) "clojure"))
(#object[java.net.URL 0x27a7ef08 "jar:file:/Users/borkdude/.m2/repository/org/clojure/data.json/0.2.6/data.json-0.2.6.jar!/clojure"] #object[java.net.URL 0x280e8a1a "jar:file:/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar!/clojure"] #object[java.net.URL 0x11e33bac "jar:file:/Users/borkdude/.m2/repository/org/clojure/core.async/0.6.532/core.async-0.6.532.jar!/clojure"] ...)
#2020-09-0415:41borkdudeThere is still nothing like $HOME or ~ in tools.deps, am I right?#2020-09-0415:41ghadicorrect#2020-09-0415:42borkdudeok#2020-09-0415:46borkdudeThe reason I'm asking is, I'm considering sort of this for clj-kondo's config resources:
:config-paths ["../base/clj-kondo", "~/dev/clj-kondo", "~/.m2/repository/rum/rum/0.12.3/rum-0.12.3.jar!/clj-kondo"]
and maybe clojure could help create that path somehow using deps.edn. Still a bit fuzzy on this.#2020-09-0415:49borkdudeI think the users would invoke clojure -Spath for their project and clj-kondo would then scan this classpath for clj-kondo top level dirs, spitting them out to some file which it uses for future reference#2020-09-0415:50borkdudebut then they'd still would add their $HOME based paths manually, which is probably fine#2020-09-0415:56alexmillernot going to interpret ~#2020-09-0415:58alexmillerI think you should still consider alias data more, since it has existing sources and merging#2020-09-0415:58alexmillerand that data could point to files#2020-09-0416:00borkdudeOh, the :config-paths will be part of .clj-kondo/config.edn, not of deps.edn. I was just thinking how clojure (or other tools) could help, since some of the config could come from libraries#2020-09-0416:57dominicmDo ns-aliases work like :alias/foo?#2020-09-0416:57alexmillerno, they are aliases like namespace aliases#2020-09-0416:58alexmillerit's a map of alias to namespace, and you can use the alias in place of the namespace in your function name#2020-09-0416:58dominicmOhh, it's not for keys in the arguments. It's for the function name.#2020-09-0416:58dominicmI see now "symbol" in the reference, my mistake 🙂#2020-09-0416:59alexmillerright. in your deps.edn {:aliases {:mine {:ns-aliases {foo my.other.thing}}}} and then clj -X:mine foo/bar to invoke my.other.thing/bar#2020-09-0416:59alexmillerand may be used for other things in the future#2020-09-0417:02dominicmCuriosity getting the better of me, why aren't aliases referenced like #ref or [:alias :mine] or something that would distinguish it as a value for reference? That would allow for :exec-args to reference aliases, or the CLI.#2020-09-0417:04alexmillersorry, not sure I'm getting the use case#2020-09-0417:05alexmiller:exec-args can reference aliases, but it's up to your program to do that resolution (via the injected basis)#2020-09-0417:06alexmillerI have thought about some kind of automatic resolution and using a tagged literal for that might be a good idea, but not ready to commit to that yet#2020-09-0417:58dominicmThis is pretty much what I'm asking about. I don't want my code to be dependent on aliases necessarily, then I have to wrap everything... It just seems like there's pitfalls.#2020-09-0417:06seancorfield@alexmiller I'm really surprised at the removal of -R and -O since those have been around since day one, as well as the removal of several -S options. I can imagine that breaking some people's scripts (it definitely breaks a couple of ours at work). And the impending removal of the ability of -A to run :main-opts is also going to break a bunch of our workflows at work.#2020-09-0417:07alexmillerinterested in that feedback if others share it#2020-09-0417:08alexmillersome of those could be made to work again for some period of time if widely desired#2020-09-0417:08seancorfieldWe have a helper script that you pass a list of aliases into and it runs clojure -A:those:aliases in multiple directories -- now that script needs to know whether/when to use -M instead of -A based solely on whether the alias include :main-opts... which... I mean how would it know?#2020-09-0417:08alexmillerlength of "period of time" tbd as well#2020-09-0417:09alexmillercan you take a look at frequency of that?#2020-09-0417:09seancorfieldWhat do you mean "frequency of that"?#2020-09-0417:09alexmillerare 95% :main-opts or 50%?#2020-09-0417:10alexmilleror 5%#2020-09-0417:11seancorfieldPretty much any library/project out there built on deps.edn that includes instructions for running tests is going to suggest using -A:test or -A:test:runner and that would be broken if -A can't run :main-opts#2020-09-0417:12seancorfieldSee https://github.com/cognitect-labs/test-runner#usage for example -- and almost any project using that test runner is going to assume that usage.#2020-09-0417:12alexmilleryes, that's why it still works#2020-09-0417:12alexmillerbut suggests to use -M instead#2020-09-0417:13alexmillerand only at some later time tbd will stop working#2020-09-0417:14alexmillerthis is step 1 of a multi step plan towards a different place#2020-09-0417:14alexmillerbut, also I'm trying to gauge the level of turmoil this will introduce :)#2020-09-0417:15seancorfieldI think changing -A will introduce a lot of turmoil 🙂#2020-09-0417:18alexmillerI await more feedback :)#2020-09-0417:51seancorfield@alexmiller I just checked, comparing behavior between stable (1.10.1.561) and prerelease (1.10.1.672) and there's no compatible way to specify arbitrary combinations of aliases that may or may not include :main-opts in a way that avoids the warnings from -A encountering :main-opts but still includes all the non-`:main-opts` deps. In other words, you can't provide a single command-line that works across stable/prerelease CLI versions the same way, without warnings, as far as I can tell? In stable, -M only brings in :main-opts so you either need -A or repeat the aliases with both -R and -M; in prerelease, if you use -A you get the warning and you can't use -R. Is that a correct reading of the expected behavior?#2020-09-0417:52borkdudeWhat will be new the new equivalent for clojure -R:foo -C:foo -Spath? So getting the classpath with alias foo deps and paths included?#2020-09-0417:52seancorfieldIt seems like -M works as a -A replacement in prerelease but that can't be used on stable (because -M doesn't bring in general alias content in stable)?#2020-09-0417:56seancorfield@borkdude I don't think that's possible in the prerelease version? Since your only options are -A (which will still run :main-opts), -X, or -M...#2020-09-0417:59seancorfieldWell, I think you can say clojure -Spath -X:foo but there's no command that is compatible across multiple clojure versions?#2020-09-0417:59borkduderight, but what if you need that? I know I'm using that now sometimes#2020-09-0417:59seancorfield(you can't say clojure -X:foo -Spath since it tries to interpret -Spath as a function to be executed)#2020-09-0417:59dominicmI use -R/C also when I have an alias with main opts, and dependencies and I only want the deps#2020-09-0418:00borkdudehttps://github.com/borkdude/clj-kondo/blob/7364c6a5dc36904c3c5e8edb3d95e0b9c33e8f0c/script/diff#L5#2020-09-0418:08alexmillerif you want just parts of an alias, you should break that into multiple aliases and combine the parts you want#2020-09-0418:08alexmillerso make an alias for the deps, and an alias for main-opts, and use either one or both#2020-09-0418:09alexmillerfrom what I can tell, the number of people that use this very targeted usage of -R/-C is probably mostly the people in this conversation :)#2020-09-0418:11borkdudemaybe we represent thousands of silent users struggling with this behind their corporate proxies ;)#2020-09-0418:11borkdudej/k#2020-09-0418:11seancorfield@alexmiller Am I correct that -M in stable does not include other stuff from the alias(es) but in prerelease it does include them?#2020-09-0418:11alexmilleryes#2020-09-0418:12seancorfieldI was hoping there was a single command that could be used across multiple CLI versions (without warnings), but that seems not possible.#2020-09-0418:12alexmillerI don't think so. I want you to move to the new thing. :)#2020-09-0418:12borkdudeso in summary: decomplect your aliases and move on?#2020-09-0418:12seancorfieldEvery project generated by clj-new so far is going to run into this (because of the :test:runner combination).#2020-09-0418:14alexmillerbecause -A:test:runner uses :main-opts and produces the warning?#2020-09-0418:14seancorfieldYup.#2020-09-0418:14seancorfieldIn particular, :runner adds the Cognitect runner and its :main-opts.#2020-09-0418:14alexmillerwell the solution is to tell them to use -M:test:runner which is where I want people to end up#2020-09-0418:15seancorfieldBut -M:test:runner will not work on stable CLI today.#2020-09-0418:15alexmilleror even better to coax tools into providing -X friendly entry points#2020-09-0418:15alexmillerthere may need to be a period of dual instructions#2020-09-0418:17alexmilleranother option would be to leave -A alone, but basically deprecate it entirely and only doc other stuff going forward#2020-09-0418:17seancorfieldRemoving -R/`-C` completely means there is a non-zero amount of tooling that cannot work at all across multiple CLI versions.#2020-09-0418:18borkdudeI think that would be the sanest: don't remove existing things unless you absolutely have to, undocument them and then wait for a year or so#2020-09-0418:18alexmillerthere is a limit - leaving existing things complicates the impl. some of that's worth it, some isn't.#2020-09-0418:19alexmillerI think very very few people use -R and -C#2020-09-0418:19borkdudebut a lot of people are using -A#2020-09-0418:19alexmilleryes#2020-09-0418:20alexmillerand it continues to work#2020-09-0418:20alexmillerwith the suggestion to do something else :)#2020-09-0418:20borkdudeprinting a warning / hint is a good thing too imo#2020-09-0418:20alexmillerit does#2020-09-0418:21alexmilleryou said above Sean "cannot work across multiple versions" but -A does#2020-09-0418:22borkdudeAt my job they are just starting to adopt deps.edn. I was away for 1.5 week and my colleague starting porting other projects to deps.edn, after I did only our front-end. He's even using -X. I told him that's only release candidate material ;)#2020-09-0418:22alexmillerhe's a man of the future#2020-09-0418:22borkdudeHe also used your boot deps edn thing @seancorfield, since we still rely on boot for most of the stuff, but slowly migrating away#2020-09-0418:23alexmillerI'm hoping it will soon be in the stable version#2020-09-0418:23borkdudeso I hope the breaking changes won't affect their initial impression#2020-09-0418:24borkdudebut if warnings are printed, I think it's fine#2020-09-0418:25seancorfield@alexmiller So what is the recommended replacement for clojure -R:foo -C:foo -Spath ?#2020-09-0418:25borkdudeSean: he already told: break foo up into multiple aliases and use -A#2020-09-0418:26borkdude(I think: https://clojurians.slack.com/archives/C6QH853H8/p1599242896182500)#2020-09-0418:26seancorfieldSo this is accepted as a breaking change? That will force "lots" of users of the CLI to modify their deps.edn files?#2020-09-0418:26alexmillerclj -Spath -M:foo -P#2020-09-0418:27seancorfieldBecause -P suppresses :main-opts?#2020-09-0418:27alexmillerit suppresses execution#2020-09-0418:27borkdudethat's also slightly confusing to me, always has been, like -A:foo -Spath.... is it going to execute main or not... I always have to try it since I can't remember#2020-09-0418:28alexmillerit may be that -Spath should imply -P, I haven't thought about that much#2020-09-0418:28seancorfieldBut there's no compatible equivalent that works across both stable and prerelease?#2020-09-0418:28alexmillerI will make a note to look at that next week#2020-09-0418:28alexmillerlike I said, hadn't thought about it#2020-09-0418:28Derek PassenWhat’s the reasoning for -Spath to continue to exist but -Stree to move to -X:deps tree?#2020-09-0418:29seancorfieldBecause -Spath is handled in the script and doesn't need to run a program.#2020-09-0418:29Derek PassenThank you#2020-09-0418:29borkdudethat's not entirely true though? -Spath uses the tools jar which is a JVM program#2020-09-0418:30borkdudeif you have a cache, then it's indeed script only, I get the difference now#2020-09-0418:30seancorfieldOnly if it isn't cached -- it's related to just the first "part" of the CLI: build classpath etc.#2020-09-0418:30alexmilleryeah, what Sean said#2020-09-0418:31alexmillerit seemed silly to pass a path to a Clojure program so it could print it :)#2020-09-0418:31alexmillerwe originally had it in the list of things to change but just didn't make sense (-Sdescribe is similar)#2020-09-0418:31seancorfieldI think my biggest concern here is that several things that work in stable will simply break in the next version.#2020-09-0418:31borkdudeI view clojure as a tool for creating classpaths, and incidentally also executing them ;)#2020-09-0418:32alexmilleryes, there are breaking changes#2020-09-0418:32seancorfieldWhich means that if you use CLI tooling across a team and across various servers/CI/etc, you have to have a coordinated upgrade of the scripts, JARs, and all the changes to your code/tooling.#2020-09-0418:33alexmillerI've spent a week deciding which migration strategy to take on each of these settings. I tried to make what I judged to be the common things continue to work#2020-09-0418:34alexmillernone of these is a final decision, that's why it's out here as a prerelease for feedback#2020-09-0418:34alexmillerthe only thing I've heard that is actually breaking you now is the -Spom I think?#2020-09-0418:35alexmillerwell let me flip it and say, please give me a list of what is breaking you#2020-09-0418:36seancorfieldIt breaks our "outdated dependency" script, which relies on -Stree and now there's no compatible way to run that script across multiple versions of the CLI.#2020-09-0418:37alexmillermaking -Stree or -Spom continue to work is an easy thing to do#2020-09-0418:38alexmiller(by internally rewriting as -X)#2020-09-0418:39seancorfieldWe don't use -R/`-C` but that will be a breakage for others with no compatible way to do it across multiple versions of the CLI.#2020-09-0418:39alexmillerI don't believe there are more than a handful of people doing that#2020-09-0418:39alexmiller(but I will be watching to see if I'm wrong)#2020-09-0418:40borkdudeFalse if old, true if new, just dropping this here :)
$ clojure -Sdescribe | bb '(-> *input* :version (str/split #"\.") (->> (mapv #(Integer. %))) (compare [1 10 1 672]) nat-int?)'
false
#2020-09-0418:41alexmillerwhat am I reading?#2020-09-0418:41seancorfieldCLI version sniffing 🙂#2020-09-0418:42alexmilleroh, ok#2020-09-0418:44alexmiller@seancorfield do you use -Stree with other args like -A/R/C?#2020-09-0418:48seancorfieldWe use it with -A#2020-09-0418:50seancorfieldCurrently, we use -A everywhere in our scripts that drive CLI stuff, because we have :main-opts in "separate" aliases -- but those :main-opts are mostly accompanied by :extra-deps to bring in deps for those mains.#2020-09-0418:50seancorfieldThe latter means we can't switch to anything that portably runs across multiple CLI versions that doesn't also produce warnings that might interfere with any output parsing we might do.#2020-09-0418:52borkdudeWhat about a CLJ_VERSION variable that invokes the right version of your CLI tool?#2020-09-0418:52alexmillerif the warnings were all stderr, would you not care?#2020-09-0418:53alexmiller(right now they are not all stderr, but they could be)#2020-09-0418:53seancorfieldWell, at this point I'm now considering putting clojure and the tools JAR into our repo under version control so we can guarantee a given version everywhere, even if a developer or server environment has a different version of the tooling...#2020-09-0418:57alexmillerfyi, there are two jars now :)#2020-09-0418:57alexmiller-X uses a bootstrap program that does not have any other dependencies and that is in its own jar as of this release#2020-09-0418:58seancorfieldYeah, a couple of our servers are old enough that I can no longer run the CLI install shell script on them so any upgrades I do on those servers is manual and that was a bit painful because there's more than just a single script to update.#2020-09-0418:58seancorfieldlein/`boot` both used wrappers that auto-downloaded the JARs if they were missing.#2020-09-0418:59seancorfieldBut making that work reliably across every environment, including older envs, is tough -- and not good for controlled corp envs with firewall restrictions etc.#2020-09-0419:00seancorfieldAnd moving this into our repo means we can also move all our tooling to leverage the new features faster without worrying about other devs or envs having older versions of the CLI stuff. So that would be a win, I guess.#2020-09-0419:01borkdude@seancorfield I'm doing the same with deps.clj (downloading the jar that is). It also has support for proxies.#2020-09-0419:02borkdudeI'm considering adding download instructions when it fails for some reason. I'm not sure if I'm allowed to bundle those jars, so I'm doing it that way mostly for that reason#2020-09-0418:53seancorfieldThat's never been a problem before because there have not been any outright breaking changes.#2020-09-0418:54borkdudeI'm willing to consider something for deps.clj which allows compatibility with both versions#2020-09-0418:54borkdudebased on some env var#2020-09-0418:56seancorfield(we used to keep the lein and boot scripts under version control to avoid these sorts of problems)#2020-09-0418:56borkdudeI guess that also works#2020-09-0419:03alexmillerI'm taking off - have a good weekend all. more feedback is fine, just won't see it today :)#2020-09-0419:03borkdudeHave a good weekend!#2020-09-0419:05seancorfieldThe lack of "portable" CLI commands/instructions across multiple (unknown) versions of the tooling is definitely my biggest concern about this...#2020-09-0419:06borkdudeIs the issue that the alphabet is too short to use different letters? What about longer option names?#2020-09-0419:08seancorfieldI don't think that solves anything @borkdude -- -M is the natural naming to choose here. It's just the transition between versions that is going to be so painful.#2020-09-0419:09borkdudeI guess especially on CI where you can't interactively react on a warning, yes#2020-09-0419:10borkdudeChecking in the CLI script is a good idea, probably we should also adopt it.#2020-09-0419:15seancorfieldSince clj-new generates new projects for people and those include instructions on how to run tests etc, I'll have to "dual instruction" all of those...#2020-09-0419:16seancorfieldOld model: -A:test:runner
New model: -M:test:runner
New command doesn't work on old CLI (because -M doesn't pull in :extra-deps) and old command produces a warning on new CLI and users -- especially beginners -- will complain that clj-new is "doing something wrong"...#2020-09-0419:35seancorfieldclojure -A:test:runner -Spath works on both stable and prerelease because the -Spath option prevents execution -- but the prerelease produces a warning about switching to -M, but if you do what the warning says: clojure -M:test:runner -Spath treats -Spath as an argument for clojure.main#2020-09-0815:44alexmillerI didn't see this go by before but order matters here - the exec opt should be last (same thing you mention later with -X) because in both cases you are passing arbitrary args to another program#2020-09-0815:45alexmillerI understand why that's particularly confusing in this example#2020-09-0816:53seancorfieldclojure -Spath -A:test:runner -- works on stable; works on prerelease but produces a warning (use -M)
clojure -Spath -M:test:runner -- works on prerelease (so switching -A to -M per the warning behaves "as expected")
clojure -A:test:runner -Spath -- works on stable; works on prerelease but produces a warning (use -M)
clojure -M:test:runner -Spath -- does not work on prerelease (so switching -A to -M per the warning "stops working")
It's probably not going to be obvious to a user why prerelease is more sensitive to order than stable.#2020-09-0816:53seancorfieldIf -Spath implied -P, I think that would provide fewer surprises.#2020-09-0817:26alexmillerYeah#2020-09-0817:26alexmillerI took a note of that#2020-09-0919:41alexmillerhave thought about this more. -Spath does already have the same effect as -P, I think the issues here are the warnings (I don't actually think those are an issue) and the ordering constraint. I could probably detect the -M case but I'm not sure it's actually worth doing so.#2020-09-0919:43seancorfieldMaybe update the docs around -Spath to make it clear that it should be before -M or -A on the command-line?#2020-09-0919:45alexmillerthe ordering is explicit in the docs etc already#2020-09-0919:46alexmillerclj-opt is always listed first and exec-opts listed last#2020-09-0919:47alexmillermight be someplace it could be made clearer#2020-09-0919:47seancorfieldAh, I had certainly missed that subtlety...#2020-09-0919:49alexmillerI'll review next time I update docs#2020-09-0919:49seancorfieldI suspect that I'd gotten used to it working on stable with -Spath at the end#2020-09-0919:51seancorfield(and, indeed, everywhere I've ever used -Spath it's at the end -- in all my CircleCI configurations and test runner scripts, and even in READMEs where I'm illustrating a usage of -Spath!)#2020-09-0419:37seancorfield$ clojure -Sforce -A:test:runner -Spath
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
/Users/sean/.m2/repository/org/apache...
so you think OK,
$ clojure -Sforce -M:test:runner -Spath
Unknown option: "-S"
Unknown option: "-p"
Unknown option: "-a"
Unknown option: "-t"
Unknown option: "-h"
USAGE:
clj -m cognitect.test-runner <options>
🙂#2020-09-0419:41seancorfield(don't get me wrong: I think the changes overall are good and the direction is good -- it's just the breakage and potential confusion that I think is going to be very problematic)#2020-09-0421:44lreadMy 2 cents, fwiw, is that breakage is totally acceptable for an alpha release. Since so many folks are relying on this alpha release component, what would be nice, where possible, is clear guidance. For example I just tried the following with v1.10.1.672:
> clojure -Spom
Option changed, use: clj -X:deps mvn-pom
My usage broke, but the fix is ultra clear.
Also since most folks just install Clojure latest with their package managers, it might be nice to highlight how to stick with the previous release until they can schedule migration. This might not help with CI tools that install the latest automatically, so a good amount of forewarning would likely be appreciated so that CI scripts can be updated.
And even though you have my 2 cents, there are sure to be folks who don’t follow Clojure closely, and don’t fully realize they have been using a changing alpha component (they knew they were using 1.10.1 but didn’t really realize they were using 1.10.1.xxx) and will be surprised because Clojure is very much known and valued for not breaking on update.#2020-09-0422:19seancorfield@lee I agree that tools.deps.alpha itself is clearly alpha and subject to change, but I think a lot of people would read https://clojure.org/guides/getting_started and not consider the CLI itself to be alpha and therefore not expect it to "randomly break stuff".#2020-09-0422:22seancorfieldAnd if two different beginners have installed the CLI at different times (one before this change and one after this change), their experience following instructions out there on the web are going to be confusingly different.#2020-09-0422:24seancorfieldThis will be compounded if they try to follow tutorials out there about the CLI and create new projects (with clj-new), they'll also have confusingly different experiences...#2020-09-0422:27seancorfieldI sympathize that the path from where we are today (at least on the stable CLI) to where we want to be "tomorrow" (when the prerelease becomes stable.next) is complicated by retaining backward compatibility here...#2020-09-0422:30dominicmUnrelated, is it/will it be considered to specify extra-deps via an alias map? Between that & paths, it would abate most complaints about copy/paste between aliases / remembering which combination of aliases to compose together.
Clasically it's :test:run because :run-tests requires copying the paths/deps from :test and people aren't so keen on that.
Not something I'm looking to drive, but curious about how that would impact future endeavours of mine.#2020-09-0422:32seancorfieldI think leaving -A alone -- retaining exactly its stable behavior, without even printing a warning message -- and just removing it from the documentation might be the best path. And perhaps at some future date adding a printed warning about using an undocumented option. That will allow a lot of tutorials etc to continue working -- without any potentially confusing warnings -- and let new/updated tutorials start talking the new options (but with caveats about CLI versions, to cater for folks who have older installations).#2020-09-0422:33lread@seancorfield and @borkdude I too empathize with folks who will be surprised by any cli breakages. And agree that it is likely not clear to the average Clojurian that they have been using an alpha component.#2020-09-0422:35seancorfield@dominicm I already have :test:runner so I can use :test in various contexts without needing Cognitect's test runner, but today -A:test -M:runner doesn't work with that setup (since -M today does not bring in :extra-deps). So either have to split :runner in two and use something like -A:test:runner -M:run-tests to avoid warnings and be compatible across both versions or provide dual instructions that are version specific (`-A:test:runner` today, -M:test:runner "tomorrow").#2020-09-0423:03nateJust another vote for what @seancorfield is saying. I use -C for mixing in a classpath into a -A invocation. I see how this can be changed for the next stable version, but having a window of deprecation would really be nice.#2020-09-0506:42seancorfieldReading over the prerelease Deps/CLI reference again, and reviewing the discussions above, it seems the -A option is being promoted as the way to do everything-except-execution whereas before it used to include execution. Given that it's being referred to as the "REPL option" and, for the most part, -R used to be somewhat similar to -A without :main-opts, I wonder if keeping -R but making it also pull in the classpath stuff (but not :main-opts) would be the less disruptive path?#2020-09-0506:42seancorfieldThen -A could quietly be undocumented, eventually deprecated, and finally removed.#2020-09-0506:44seancorfieldThis would be a minor expansion to what -R currently does (it would respect :extra-paths which only -A/`-C` did before) and it would still ignore :main-opts. I suspect -C is the least used option and almost all -A uses could be replaced by either -M (in the new world) or -R (in either world) with no changes in behavior.#2020-09-0506:45seancorfieldAnd -R more closely relates to REPL. So we'd have -R/REPL, -X/execute, and -M/main.#2020-09-0506:50seancorfieldHmm, I guess -R would also have to subsume -O for that to work fully.#2020-09-0506:53seancorfieldGoing back to the :test:runner example, it seems like -A:test:runner -M:runner works identically on both stable and prerelease and produces no warning on prerelease (presumably because -M is present?).#2020-09-0506:54seancorfieldUgh! So... I guess instructions could change now to recommend -A... -M... and that would work now and tomorrow even though it's a bit redundant.#2020-09-0507:05borkdude@seancorfield I like your suggestion of extending R to include C, I've always found it weird that these two were separated. Also the -A -M trick is something I tried yesterday, but somehow didn't work for me, since I got file not found -M, but now that I'm trying it again, it seems to work.#2020-09-0507:08seancorfieldYeah, having integrated CLI/t.d.a into our repo and started to update all our script to match this "new world", I'm very torn between "extending -R and undocumenting -A" and just "documenting -A... -M..." as the preferred approach.#2020-09-0507:09seancorfieldI would much prefer a solution that works for today and tomorrow (which is the -A... -M... approach, especially since that suppresses the warning on -A).#2020-09-0507:10seancorfieldEither way, our build script that previously just turned build aliases subproject options into cd subproject; CLJ_CONFIG=../versions clojure -A:aliases options needs to be way smarter about handling the options part...#2020-09-0507:10borkdudeYeah, I guess clojure could override certain deprecated options when the more preferred ones are used, which provides a migration path#2020-09-0507:10seancorfieldWell, it already does.#2020-09-0507:11borkdudeYes, but also for other deprecated options?#2020-09-0507:11seancorfieldIf you use -A:test:runner -M:runner you don't get the warning, even when :runner includes both :extra-deps, :extra-paths, and :main-opts.#2020-09-0507:11seancorfieldWhich deprecated options?#2020-09-0507:11borkdude-R, etc#2020-09-0507:11seancorfield-R, -T, and -O aren't deprecated, they're simply removed.#2020-09-0507:12seancorfield-A's use of :main-opts is deprecated.#2020-09-0507:12borkdudeI've never used -T or -O, so R is what I'm concerned about. So if you use -R ... -A ... in the new world, does that do the same as in the old world?#2020-09-0507:13seancorfield-R cannot be used in the new world. It does not exist.#2020-09-0507:13borkdudebut is it ignored?#2020-09-0507:13borkdudeas in, no error#2020-09-0507:13seancorfieldIt spits out an error and halts the script.#2020-09-0507:13borkdudethat's what I meant#2020-09-0507:14seancorfield(! 2172)-> clojure -R:test -A:test
-R is no longer supported, use -A for repl, -M for main, or -X for exec
Sat Sep 05 00:13:57
(sean)-(jobs:0)-(/Developer/workspace/wsmain/build)
(! 2173)-> echo $?
1
#2020-09-0507:14borkdudebut I guess -A would also override -R in the old world, so in the old case it also doesn't work as a polyfill for both#2020-09-0507:14seancorfieldRight, the problem is that -A in the old world also executes #2020-09-0507:15seancorfieldBut at least changing -A:... to -A... -M... works identically and doesn't issue a warning in either world.#2020-09-0507:15borkdudethat's already a win#2020-09-0507:15seancorfieldBut anything that uses -R today is plain ol' broken tomorrow.#2020-09-0507:16seancorfieldThere's no compatible way to run anything that uses -R today in a way that works regardless of clojure version I think.#2020-09-0507:16borkdudeI have a fixed version on CI, so I don't think it will break for me:
https://github.com/borkdude/clj-kondo/blob/70b3e54130978692c61e53859b126ffbe8a492f0/.circleci/config.yml#L24#2020-09-0507:17seancorfieldSo you pin to an old version?#2020-09-0507:17borkdudeI never thought about it really hard, just copied the install instructions and that seems to contain a version number#2020-09-0507:18seancorfieldFine for CI for you, but not fine for any library code or tools that have to live in a version where members could have any version of CLI installed...#2020-09-0507:18borkdudeI do have one custom installer script for mac CI as the linux script didn't work there:
https://github.com/borkdude/clj-kondo/blob/master/.circleci/script/install-clojure#2020-09-0507:18borkdudeBut in that arena people see warnings and can react to them#2020-09-0507:19borkdudewhich can still be confusing, no argument there#2020-09-0507:20seancorfieldI have a solution for clj-new now so I'm somewhat mollified 🙂 but it doesn't help anything that uses -R and has to live in a multi-version world.#2020-09-0507:21borkdudeI guess clojure could print a link to a more elaborate blog post about the situation#2020-09-0507:21seancorfieldI don't consider that a solution. "We broke your shit. Go read this to fix it!"#2020-09-0507:22seancorfieldSeriously, that is not the right approach, esp. given Rich's whole pulpit-bashing about no breaking changes without renaming things.#2020-09-0507:24borkdudeI agree that it's far from ideal, but assuming that it's going to be like it is now, it provides people a place to read up if they are curious about why it changed.#2020-09-0507:25seancorfieldConsidering how many people learn Clojure by using lein and still trip over ~/.lein/profiles.clj stuff, I don't think we should just throw our hands up and accept this and just say "Well, read these docs about why we broke stuff". I really don't.#2020-09-0507:27borkdudeYou don't have to convince me that avoiding breakage would be preferred. I think I'm going to take a break from this channel for the rest of the day :)#2020-09-0507:27seancorfieldI can change all the -A uses in clj-new's docs and in the generated projects to be -A... -M... and at least I then say, "Hey, the latest version does recommend using both options so you're future-proofed" but it's still a fairly sucky experience for users.#2020-09-0507:28seancorfieldEnjoy your weekend @borkdude 🙂#2020-09-0507:44seancorfieldProposal:
* Recommend authors change -A:stuff to -A:stuff -M:stuff for compatibility with both versions where they want :main-opts executed.
* Keep -R and expand it to include :extra-paths and :jvm-opts so it mnemonically matches REPL: -R/REPL, -X/eXecute, -M/Main.
* Undocument -A for now, officially deprecate it later (and add that warning about using -M instead), and eventually drop it altogether.#2020-09-0514:25alexmillerI think this is pretty good. I'll have to go back through my spreadsheets to evaluate more. I think the first bullet recommendation could even be for tool authors to do nothing, wait until most people are using a new version, then change their recommendation.#2020-09-0514:30seancorfieldIf you don't introduce the warning for -A -- per bullet 3 -- then, yes, bullet 1 isn't really necessary.#2020-09-0514:32alexmillerthe warning is what tells people how to move to the new world - without that, won't people be stuck on -A forever?#2020-09-0514:33seancorfieldBullet 3 suggests introducing the warning some time later, after everyone has had a chance to update.#2020-09-0514:35seancorfieldPeople don't like warnings from their day-to-day development tools. Beginners in particular will complain to library authors about "incorrect instructions" if you introduce that warning before anyone has had a chance to update instructions/tutorials.#2020-09-0514:36alexmillerI'm ok with that :) I'm not sure I'm in agreement with you on the scope of this as an issue#2020-09-0514:37alexmilleryou're right that they don't like warnings, so they'll change what they do, which is what we want#2020-09-0514:38alexmillerlibrary authors can either do nothing and situation is same
or they can use dual instructions for a time
or they can doc new instructions and say clj > X.Y.Z (which also gets people on new version)#2020-09-0514:38alexmilleror they can start using -X style invocation (which also drives to new version)#2020-09-0514:39seancorfield> I'm not sure I'm in agreement with you on the scope
I'm pretty sure we're just not in agreement on that 🙂#2020-09-0514:39alexmillerthere is a middle ground - no warning for -A on first stable release#2020-09-0514:39alexmillerthen warning#2020-09-0514:39alexmillerthen (much later?) removal#2020-09-0514:41seancorfieldHaving been on the receiving end of complaints about incorrect instructions (or version-specific instructions that weren't absolutely clear on how to determine the version and how to "solve" the problem, on all platforms), I'm perhaps more sensitive to this whole issue than you... 🙂#2020-09-0514:41seancorfieldYes, bullet 3 is that middle ground.#2020-09-0514:41alexmillerI'm on the receiving end of those complaints too#2020-09-0507:45seancorfieldThis provides a migration path for the most common use cases that is minimally disruptive (most uses of -R today won't care about paths or JVM options I suspect). It improves naming for the three desired use cases: REPL, eXecution, and Main opts. It breaks the least amount of code out there.#2020-09-0507:50seancorfield(question for Alex when he gets back: why are there two identical copies of exec.jar in the CLI distribution -- one at the top level, one in libexec?)#2020-09-0514:31alexmillerI think there aren't two in the distribution, the install script copies the root one to libexec which is the structure brew expects, although I think maybe I didn't sync the clojure script up to that and it's using the root one. I'll check on that#2020-09-0516:03seancorfieldFYI, confirmed that script uses the root one:
-classpath "$cp:$install_dir/exec.jar"
#2020-09-0514:28alexmillerKeep in mind that most Clojure users are on Mac and install via brew and that brew aggressively updates you to latest version (our stable) even if you're upgrading something else. So within some period of time (couple months maybe), many people are using newest version.#2020-09-0514:28alexmillerKeep in mind that most Clojure users are on Mac and install via brew and that brew aggressively updates you to latest version (our stable) even if you're upgrading something else. So within some period of time (couple months maybe), many people are using newest version.#2020-09-0515:27practicalliThat only covers 55.36% of the survey responders, so this doesnt seem to be the strongest of arguments.
Of course I am happy to help encourage upgrading to newer versions in any way I can, once I've understood the changes myself.#2020-09-0515:31seancorfieldI wonder what percentage of the 35%-ish that are on Linux are using brew vs manually installing via the .sh file?#2020-09-0515:36practicalliThere are legion of package managers for Linux that brew competes against, maybe a question for the next survey?
I find it easier to use the Linux script install.#2020-09-0515:51seancorfieldI use brew on Ubuntu (on WSL) because it's easy to keep clojure up-to-date that way but that's recent for me -- I used to use the manual install script, but I tended to only upgrade when I remembered to do it, or I tried to do something that my older, installed version didn't support 😐#2020-09-0521:18dominicmI've used Linux for many years, the brew stuff is extremely rare.#2020-09-0521:24seancorfieldYup, I'm just glad it exists for clojure. Getting stuff into the more standard installer repos on Linux seems very laborious? What relatively painless options exist for Linux package managers that the Clojure core team could leverage @U09LZR36F?#2020-09-0521:25dominicmHa! 😂#2020-09-0521:26dominicmIf only you knew the rabbit hole you'd just asked about#2020-09-0521:27dominicmEvery distribution has its own rules and timelines.
Consider Ubuntu LTS. They don't take changes. Canonical pay people to backport security patches. Every now and then you get a system upgrade (like mac) and that's when you get changes. New features, bug fixes, etc.#2020-09-0521:28dominicmThere are, of course cutting edge distributions like arch and void. They'll update whenever the package maintainer feels like it. Usually regularly for important changes.#2020-09-0521:29dominicmSome distributions care about reproducibility. They have other constraints I don't understand.#2020-09-0521:30dominicmI am a void clojure package contributor, and my life is pretty easy. The installer script was "ported" to the void build system in ~10 lines and it's just worked since then.
Maybe I'll need to make changes with the new system.#2020-09-0600:02practicalli@seancorfield To package Clojure for Ubuntu, then I'd suggest a personal package archive (PPA), new release of Clojue can be pushed to this as soon as they are ready. There is a specific process for creating package to learn. Packages can be built for each release of Ubuntu (e.g. current long term support and intermediate releases) and published on the PPA. The relevant packages can be submitted for each new Ubuntu release (6 month cycle).
There are also other options such as snaps and flatpack, but a proper *.deb package is much nicer in my opinion as it ensures there are no conflicts with other software installed via the package manager. There is a Leiningen package for Ubuntu, so some of the experience from that can be used.
For Arch, I seem to remember that @U11EL3P9U packages up clj-kondo , so I assume his work can be of help for packing to the Arch User Repository (AUR).#2020-09-0605:43dharriganThat's right, I do maintain a few Clojure packages on Arch, like clj-kondo, babashka, clojure-lsp. A pleasure to do so 🙂 However, the main Clojure package is part of the Community Repo that is under a different maintainership. It is reguarly updated (last update was 2020-08-26, which brought in Clojure 1.10.1.645).#2020-09-0514:28alexmillerAlso keep in mind that we are in work on a 1.10.2 Clojure cycle and we tie brew versions to Clojure versions, so that might be a natural point to either coax an upgrade or drop some kind of compatibility#2020-09-0515:56seancorfieldI have updated the depstar README to show usage with the prerelease -X option: https://github.com/seancorfield/depstar/blob/develop/README.md#clojure--x-usage#2020-09-0515:56seancorfieldIt works quite nicely that way...#2020-09-0515:59seancorfieldI expect I'll update clj-new next, to support -X usage...#2020-09-0516:34borkdude> Keep in mind that most Clojure users are on Mac and install via brew
Have you considered that those Mac users will probably also use the linux scripts on CI? Do you get any stats from that usage? It may be less of an issue since the install script is versioned, but since people tend to develop CI scripts locally (I often do), it might become confusing.#2020-09-0516:36borkdudeThere might also be less Windows users using this because for some people it's harder to get going than lein. I got this report yesterday and it's not the first time I hear these kinds of issues.
> Clojure until 1.6 used to work on Windows from https://chocolatey.org/packages/clojure too, without problems. Clojure Tools broke everything and it seems that OSX ( Homebew ) is the only "first class citizen" now.
https://github.com/babashka/babashka.process/issues/15#issuecomment-687035007
Don't shoot the messenger, I don't really have a stake in Windows support because I know how to work around it :)#2020-09-0520:32alexmillerOh I definitely think that’s a factor#2020-09-0516:49seancorfieldYeah, you have to be pretty persistent to use Clojure CLI/`deps.edn` on Windows, and willing to work with very "alpha" tooling. Which a lot of Windows users are not willing to do: they often have different expectations of the user experience than devs on macOS/Linux.#2020-09-0516:49borkdudeOne personal data point: I found it difficult to install clojure on Windows CI because the powershell stuff you had to enable and whatnot. That's why I also went with lein to run tests there:
https://github.com/borkdude/clj-kondo/blob/master/appveyor.yml#2020-09-0516:51borkdudeI do use clojure on the other two major OS CIs#2020-09-0516:53seancorfieldI'll be using Windows for development a lot more going forward (Microsoft had a great deal on Surface Laptop 3 so I just ordered an i7 with 16GB RAM and 256GB SSD). But I'll be leaning on WSL2 very heavily, and using (Linux) brew for installation.#2020-09-0517:03QuestI have an odd case -- at least from my pure Clojure background (didn't start in Java.)
I have a Java dependency that doesn't publish to maven. I can run ./gradlew build on it to get a library .jar.
What's the best way to include this into deps.edn dependencies?#2020-09-0517:06QuestCan I publish just a .jar to maven under my own repo?
I could try updating the library to properly publish to Maven & PR'ing it, but I'm just unfamiliar with the gradle ecosystem. The library author says he'll start publishing to maven soon...#2020-09-0517:08seancorfieldYou can use a local .jar file directly in deps.edn via a :local/root dependency. We use that for a couple of things that have to build locally because they aren't published anywhere.#2020-09-0517:09Questoh man, I immediately get what you mean, but how did I utterly fail to find this in a doc somewhere? I spent 20 minutes searching around#2020-09-0517:09Questanyway thanks Sean, you're a big help in many channels 🙂#2020-09-0517:09seancorfieldIt's mentioned here https://clojure.org/guides/deps_and_cli#_using_local_libraries#2020-09-0517:10seancorfieldAnd here in more detail https://clojure.org/reference/deps_and_cli#_dependencies#2020-09-0517:10seancorfieldIt doesn't "scale" so well if you're part of a team or want to use it in a CI pipeline, but it's fine for local development and testing.#2020-09-0517:11QuestAhh, I'll try just reading through the whole deps guide. I kept searching for "clojure deps include .jar library" and similar terms, never thought of "local library"#2020-09-0517:13QuestAs far as "scaling" this approach, I was just going to git submodule the dependency library.
I could see how many layers of builds would take a long time / cause odd failures. Thankfully still at "fun personal project" size for now#2020-09-0517:43Questdeps.edn didn't seem to pick up on the .jar (silent failure?) as I couldn't load any expected classes.
I realized there's no pom.xml, probably why.
At a minimum, I believe I need to tweak the library to use https://docs.gradle.org/current/userguide/maven_plugin.html#2020-09-0517:54seancorfield@U22M06EKZ You shouldn't need a pom.xml for this usage -- but if that Java library has any dependencies, you need some way to tell clojure about them (if the JAR contains a pom.xml the t.d.a machinery would take care of that, else you could just add those deps directly to your deps.edn file).#2020-09-0517:57Quest@seancorfield Hmm, I could uptake them into my deps.edn. Do you know of a way to determine if the jar was loaded or not?
I've just been trying to run an :import on a class that I expect to exist, but wondering it there's a better way to poke around#2020-09-0518:00seancorfieldUse the -Spath option on the clojure CLI to see what the classpath contains.#2020-09-0518:01seancorfieldOr even inside your REPL: (System/getProperty "java.class.path")#2020-09-0518:02seancorfielduser=> (run! println (clojure.string/split (System/getProperty "java.class.path") #":"))
/home/seanc/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/home/seanc/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/home/seanc/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
src
nil
user=>
(example from a bare REPL)#2020-09-0518:05Quest^ I figured there was something from the REPL, but -Spath is a good tip. Appreciate the knowledge-share, you had a hand in me starting adoption of deps.edn a year ago too and I doubt I would've tried it otherwise 👍#2020-09-0518:07seancorfieldCool. We've never regretted switching to CLI/`deps.edn` in 2018 at work.#2020-09-0518:08seancorfield(and we're using the bleeding edge prerelease version right now -- as of yesterday)#2020-09-0517:26practicalliFor the up-coming release of Clojure CLI tools:
I have been using clojure -R:cognitect-rebl -A:nrebl to run REBL with nREPL. The -R:congnitect-rebl load in dependencies from an alias that also has a :main-opts (which it not the main namespace to run) and -A:nrebl which has a :main-opts I do want to run.
If I create a separate alias for cognitect-repl without the :main-opts and use clojure -M:rebl-deps:nrebl it works.
Or I could just add all the rebl dependencies into the :nrebl alias.
Does that seem the right approach?#2020-09-0517:32seancorfield@jr0cket I suspect if you used -A:cognitect-rebl:nrebl it would run the :main-opts from :nrebl since they overwrite each other -- last one wins.#2020-09-0517:34seancorfieldYou could also use -A:cognitect-rebl:nrebl -M:nrebl which would work on both today's stable version and the current prerelease version without the -A warning about using -M to run the :main-opts (since the "winning" :main-opts here is using -M).#2020-09-0517:39seancorfieldI think the guidance from Alex has fairly consistently been to separate out :main-opts from other stuff if you need to mix'n'match main opts from several aliases. But the expansion of both -M and -X to respect :extra-deps and other stuff in the latest prerelease feels like the pressure is in the opposite direction -- toward bundling more things into a single alias for convenience @jr0cket Not sure how closely you've been following the discussions generally in this channel?#2020-09-0518:56practicalliI've read the whole conversation in here and reading through the docs Alex wrote. Still digesting what it all means.
I'm currently testing all the aliases I have to understand the changes#2020-09-0519:07seancorfieldI'm working on updating clj-new to work with -X, much as I've updated depstar. I'll probably add :new-x and :depstar-x aliases to my .clojure/deps.edn file so folks can use those tools via -X as examples. It'll be a while before that can become the default 🙂#2020-09-0520:28alexmiller@jr0cket would love to hear your feedback too#2020-09-0601:12practicalli@alexmiller so far very positive changes. I like that -M feels similar in concept to the :main in Leiningen, in part easing the move from Leiningen to CLI tools.
My current thoughts are around edge cases where I have multiple aliases containing :main-opts and how to use them together with certainty rather than optomism. I have tried to elaborate some options in this gist
https://gist.github.com/jr0cket/6fae7576a87d5d32eda4be2253f41692#2020-09-0601:29seancorfield@jr0cket Multiple aliases containing :main-opts is an "issue" today (stable) as well as tomorrow (prerelease), and the answer is the same as it has always been -- per the documentation -- "If multiple maps with these keys are activated, only the last one will be used" -- see https://clojure.org/reference/deps_and_cli#_aliases which explains how each different type of option is "merged" when multiple appear. The same wording is present in the prerelease reference docs, but spread out over multiple sections, rather than gathered together in one section.#2020-09-0601:30seancorfieldIs that not "certainty" enough?#2020-09-0608:02practicalliCertainly in my mind would be as part of the syntax rather than convention. If the convention changed in a release without requiring a change to the syntax of use.
Convention also means more to remember, especially as some parts of slides are merged and others only take the last. I assume (hope) nothing takes just the first.
Not saying it's an issue, but it adds to the learning curve.#2020-09-0608:11practicalliAs it's a current issue and there isn't anything in the up coming changes to resolve it, then this aspect seems a moot discuss point. I'll make a decision and move on.#2020-09-0815:52alexmillerfwiw, I don't see this as an issue - the documented behavior is what it does and what I expect it to do in the future#2020-09-0517:41seancorfield(but of course, at this point, there's still some potential for changes in the prerelease that the -A/`-M` usage recommendations may change)#2020-09-0517:42hugodI happened to watch Spec’able before reading Alex’s post. Seems pretty ironic.#2020-09-0517:42seancorfieldSpec'able?#2020-09-0517:43hugodsorry, sepc’ulation#2020-09-0517:44seancorfieldAh yes, https://www.youtube.com/watch?v=oyLBGkS5ICk#2020-09-0517:44seancorfield(transcript: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md )#2020-09-0517:51seancorfieldTL;DR: don't break stuff, only add stuff and fix bugs 🙂#2020-09-0519:13andy.fingerhutat least once you decide you are out of alpha#2020-09-0522:45hugodThe deps/cli reference page only talks about the library being in alpha. The guide doesn’t mention it at all, and besides “But, that is not to say just leave your thing 0.0.967. At a certain point, you are going to have users, and whether you change it to 1.0 or not, they are going to be depending on your stuff.”#2020-09-0519:35seancorfieldFeedback on the "WARNING: When invoking clojure.main, use -M" message (for @alexmiller) -- this usage produces that warning: clojure -m my.entry.point -- is that intentional? Do you really want folks to write clojure -M -m my.entry.point instead?#2020-09-0520:19alexmillerYes#2020-09-0520:34borkdudeIs this to discourage that usage in favor of -X? It seems to be another maybe unnecessary breaking thing that's been there for a long time#2020-09-0520:35alexmillerIt’s not to discourage it as much as that we are trying to open up the repl arg space for other things#2020-09-0520:35alexmillerAnd that means making clojure.main usage explicit#2020-09-0520:41seancorfieldInteresting choice. A lot of tutorials out there talk about running simple Clojure projects with just clojure -m my.entry.point#2020-09-0520:41seancorfieldI think beginners will (quite rightly) ask "Why do we have to specify two m options?"...#2020-09-0520:43alexmiller“Lot” = ? 2 or 3?#2020-09-0520:44alexmillerAnd if these tutorials transition into talking about -X...#2020-09-0520:46alexmillerWe want clj to do more. In some cases that means restructuring these options. It’s either that, or clj2 and clojure2 which do not seem better to me#2020-09-0520:46seancorfieldSure, if all the books and tutorials that exist today are updated to use -X instead of -m, it's not going to be a problem 🙂#2020-09-0520:52dpsuttonThanks for balancing all these things in a spreadsheet so lots of concerns are managed. On a Saturday no less. Looking forward to the previews#2020-09-0521:05seancorfield(I just want to be clear that I think all the new functionality is great -- all of my concerns are around how existing (stable) functionality is deprecated and/or removed over time)#2020-09-0521:06seancorfieldAnd, ultimately, I'm not going to be the one complaining about the changes since I'm already on the latest prerelease version on every machine and I'm aggressively adopting the new functionality!#2020-09-0521:08seancorfield(I am somewhat concerned about users of my CLI/`deps.edn`-based projects, of course 🙂 )#2020-09-0521:08seancorfieldComing in the next version of clj-new: https://github.com/seancorfield/clj-new#clojure--x-usage#2020-09-0521:09borkdudeThere was a time when the clojure CLI wasn't driven by tools.deps.alpha right? Wasn't it non-alpha back then? Isn't only tools.deps alpha but the CLI / shell script now also got alpha?#2020-09-0600:13alexmillerNo, clj and tools.deps were developed together#2020-09-0521:09seancorfield(`depstar` added support for -X in its most recent 1.1.104 release, last week)#2020-09-0521:12borkdudeI could be misremembering things, but I think there was a clojure package already years ago, before tools.deps.alpha? Not sure what it did back then, but -m was maybe already a part of it back then#2020-09-0521:14seancorfieldI know there was an unofficial clojure script floating around which behaved differently than the official one, once it appeared.#2020-09-0521:14borkdudemaybe it just did java -jar clojure.jar #2020-09-0521:17seancorfieldYeah, I think that was pretty much it. My recollection of the official CLI was that it appeared in tandem with t.d.a during the 1.9 cycle once Spec was split out of the core clojure.jar since you could no longer just run a REPL with the JAR file alone so it was supporting machinery for that... but I'm having a hard time finding any of the original posts about it (maybe I should scan back through Inside Clojure since I bet Alex blogged about it).#2020-09-0521:18borkdudeah right, thanks#2020-09-0521:19seancorfieldHmm, the pre-1.10 Inside Clojure posts are fairly sparse...#2020-09-0521:21seancorfieldThis is the first mention of t.d.a on the Clojure mailing list: https://groups.google.com/g/clojure/c/FpMqtNu0TCE/m/wc2n1tPHBwAJ#2020-09-0521:22seancorfield(July 2017)#2020-09-0523:52seancorfieldBeen playing with the new features, and it's nice that you can have the following
:new {:extra-deps {seancorfield/clj-new {:mvn/version "RELEASE"}}
:exec-fn clj-new/create
:exec-args {:template lib} ; default
:main-opts ["-m" "clj-new.create"]}
and it works with -A:new today (or -A:new -M:new) and, with the prerelease it works with -M:new (or -A:new -M:new without giving a warning) and it also works with -X:new so you can have a single alias provide for both styles of execution.#2020-09-0715:52practicalliForgetting migration for a moment, would you see :exec-fn and :exec-args replacing :main-opts in the long term?
Using these new keys seems a nicer approach for the design of the alias code itself.
Would this be an example of edn over strings that Alex mentions?#2020-09-0716:47seancorfieldI think we'll see a good uptake on the :exec-fn approach because you can avoid parsing strings to get your arguments and you can easily provide defaults -- configurable defaults, since each alias have have its own :exec-args map.#2020-09-0523:57seancorfieldThat's probably what I'll do in my dot-clojure repo by way of education for folks and as a way to encourage folks to transition (or at least see what the options available are).#2020-09-0601:18hugodI like the new features. :ns-default and :ns-aliases look very useful. The -P option is great, and could be even nicer if it (optionally) output a map of the resolved dependencies, which could be used to make reproducible builds.#2020-09-0601:40alexmillerEvery build outputs a lib map in the cache dir with this info (-Sdescribe tells you where)#2020-09-0601:49hugoddidn’t realise that 🙂#2020-09-0601:18hugodTo be honest, I’ve always found the different merging rules confusing and never really understood their purpose. Why not just have -A to specify the aliases to merge, and -e, -x (equivalent of the :exec-fn key) or -m (equivalent of the :main-opts key) to specify or override what should be run. Maybe with a -r for a future repl specifier.
If the config is not specific, define a precedence for :exec-fn and :main-opts.
This would separate (decomplect, dare I say?) merging of aliases from specifying what to invoke, which I, for one, find simpler to understand.#2020-09-0601:41seancorfield@hugod In the stable version, -M didn't pick up any items from the alias(es) except :main-opts and in the first version of -X on the prerelease track didn't pick up any items from the alias(es) except the fn/args spec as I understand it. In the latest prerelease, both of those pick up everything in the alias(es) specified and then -M runs the :main-opts while -X runs the function invocation -- and -A is slated for being similar (picking up everything in the alias(es)), but ultimately just running the REPL, not executing main/function stuff. That's why I proposed keeping But, either way, it's a much more streamlined piece of logic: all three options pick up all options and then each one of -A (or maybe -R?), -M, and -X selects what execution model is used.#2020-09-0601:41seancorfieldIt's nice and streamlined. And I think it removes a lot of confusion about which parts of an alias get picked up by which options.#2020-09-0601:50hugodHaving -M, -X, and -A pick up everything certainly seems like an improvement. I would still find it simpler to just have the one option for specifying aliases, and then options for specifying what to run. Using lower case for the latter would allow for the cli to maintain backward compatibility iiuc.#2020-09-0601:52seancorfieldI suspect there would be confusion, since -e, -m, and -r are existing main options today (for clojure.main).#2020-09-0601:55seancorfieldYou'd also need to specify both an option to collect aliases and then an additional option to specify the action to take. With the prerelease direction, you'd only need one option: either -X for aliases and execute function, or -M for aliases and main function, or -A for aliases and the REPL (at some point, given -A currently still executes main opts).#2020-09-0602:03hugodI was suggesting -e and -m kept their current meaning, and implied invoking clojure main.#2020-09-0602:04seancorfieldBut :main-opts can contain a combination of those so I don't think that's going to be intuitive.#2020-09-0602:05seancorfield(`:main-opts` can also contain -i and -r options)#2020-09-0604:59mike_ananev@alexmiller Is it possible to add functionality of generation distributionManagement section to 'clj -X:deps mvn-pom' command?
I need to distribute my artefacts written in Clojure to corporate Nexus. To do this I need section in pom.xml:
<distributionManagement>
<repository>
<id>releases</id>
<url></url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<url></url>
</snapshotRepository>
</distributionManagement>
Building and deploying artefacts is performed in corporate Jenkins, where I can't manually add this section.#2020-09-0605:02seancorfield@mike1452 Since clj -X:deps mvn-pom only updates dependencies and a few other bits, so what I tend to do is generate an initial POM, augment it manually, check it in to version control, then just run that command as needed to update the dependencies section (and it leaves all the rest intact).#2020-09-0605:02seancorfieldSo you could manually add that section, and clj will not overwrite it.#2020-09-0605:34vlaaadWeekly reminder that reading/updating pom.xml is a workaround for the problem, not a solution :)#2020-09-0607:24borkdude@alexmiller I know tools.deps isn't going to expand ~, but I'd like to know the reason for this. Reason I'm asking is that I'm considering doing this in .clj-kondo/config.edn and would like to know if I'm missing some problems#2020-09-0611:46alexmillerBecause then everyone reading deps,edn has to interpret the file#2020-09-0611:47alexmillerIt’s no longer just edn/read, it’s that plus run magic code#2020-09-0612:32borkdudeWhy does it have to be interpreted by anything else than tools.deps which already does interpretation of this entire model?#2020-09-0612:37borkdudeI mean, merge, assoc, etc still works for other tools in this case?#2020-09-0612:45alexmillerIf you have alias data, anyone may be reading it via the basis (without tools.deps, which has a lot of deps)#2020-09-0612:47alexmillerThere are a bunch of tools already reading and manipulating deps.edn. The less they have to know, the better#2020-09-0613:30borkdudeI don’t think this will be a problem for the linter config file. Thank you sir.#2020-09-0609:31mike_ananev@seancorfield Is it possible to exclude clj source files from uberjar using depstar? I need only aot compiled files in uberjar, when distributing it. Also, how do you recommend to compile java source files, when using depstar?#2020-09-0609:42mike_ananevIs it possible to add to depstar something like this https://gist.github.com/just-sultanov/e2f61734e22e2ed3981216ed9b880bc0 to compile java sources?#2020-09-0609:51borkdude@mike1452 depstar has this for removing itself:
https://github.com/seancorfield/depstar/blob/a542560766cfd2b51d415908885625aa64165cb4/src/hf/depstar/uberjar.clj#L292
I think that can be easily extended to remove other things#2020-09-0609:52borkdudeAh no, scratch that, that's only for removing something from the classpath.#2020-09-0609:52borkdudeI meant this: https://github.com/seancorfield/depstar/blob/develop/src/hf/depstar/uberjar.clj#L150#2020-09-0609:54borkdudeAh, and that's exposed via -X (not clojure's -X, but depstar's -X)#2020-09-0609:54borkdudeso probably something like -X "\.clj$" would work#2020-09-0612:53dominicmYou don't need to remove the clj files anyway. Or is there an additional piece of information?#2020-09-0618:33seancorfieldThe regex-based file exclusion was added in the most recent depstar release @mike1452 -- did the suggestions above work for you?#2020-09-0618:34seancorfield(you could also specify a classpath that only had the class path, without the src path I suppose, if every piece of source has been compiled?)#2020-09-0618:35seancorfieldAs for Java, that's outside the scope of a Clojure tool: compile it yourself and then add that to the classpath (depends on how/where you compile your Java source to .class files).#2020-09-0622:02dominicmHow can I remove org.clojure/clojure from the classpath? I seem to recall a trick to it.#2020-09-0622:03dominicm(I want to use org.clojure/clojure$sources instead)#2020-09-0622:39alexmillerJust specify a different coord for it? Or use :override-deps?#2020-09-0622:41alexmillerKeep in mind that everything calls clojure.main and you need a class for them jvm to invoke#2020-09-0622:45dominicmI tried using override-deps, maybe I used it wrong.#2020-09-0622:46dominicmError building classpath. Coordinate type not loaded for library org.clojure/clojure in coordinate {}#2020-09-0622:57dominicm(I also tried nil)#2020-09-0623:25seancorfieldYou need an alias to use :override-deps#2020-09-0623:26dominicmI used one. But I'm trying to override to remove it and {} didn't work at that removal coordinate.#2020-09-0700:46seancorfieldCan't you override it to point to local JAR? The sources JAR file?#2020-09-0701:25seancorfieldSince we're getting a lot of depstar questions here, I created a dedicated channel for support/questions/etc. Also for clj-new.#2020-09-0706:59borkdude@dominicm you can also use classpath overrides with nil. I’ve used this for creating babashka uberjars (see its README and look for uberjar)#2020-09-0709:53kirill.salykingoodmorning,
please advice how I can turn off the OmitStackTraceInFastThrow?
clojure -A:app -J "XX:-OmitStackTraceInFastThrow"
doesnt seem to work#2020-09-0709:54dharriganclojure -J-XX:-OmitStackTraceInFastThrow#2020-09-0709:54kirill.salykinthanks! it is all about the order#2020-09-0709:55dharrigannp#2020-09-0710:28practicalliIn clojure pre-release (1.10.1.672) it seems that the -X flag does not download dependencies specified in the alias :extra-deps section. Is this the correct behavior?
From my interpretation of "-X now supports multiple aliases, aliases with other argmap options like :extra-deps, and ad hoc functions" I would have thought deps would be downloaded#2020-09-0710:29borkdudeis that in combination with -A?#2020-09-0710:29practicalliNo, just by itself. I understood -A was being depreciated, so am trying to understand the scope of using -M (which seems fairly clear) and -X which I am still confused about (from a users ponit of view)#2020-09-0713:30alexmiller@jr0cket that is intended to work, so could be a bug#2020-09-0715:24practicalliYes, it does work on a clean install of Clojure CLI with the ~/.clojure/.cpcache removed. This is making much more sence now. Thanks.#2020-09-0713:45rickmoynihanTo temporarily swap out a git dep for a local dev version, am I normally better adding an alias with :override-deps in it to a :local/root, or adding :classpath-overrides to my ~/.clojure/deps.edn ?#2020-09-0713:48dominicm@rickmoynihan I usually use an override-deps (actually extra-deps because it's the same otherwise) in ~/.clojure/deps.edn because:
• Nobody else can use it - has my homedir in
• Most people on my teams aren't contributing to those other libraries anyway. It's less reusable than one would think
#2020-09-0713:50rickmoynihanyeah for both options I’m talking about adding it to ~/.clojure/deps.edn… trying to understand which way is better.#2020-09-0713:51rickmoynihanOne (`:override-deps`) effects resolve-deps and the other make-classpath right?#2020-09-0713:52rickmoynihanPractically though; aren’t they both equivalent?#2020-09-0713:52dominicmI think for your case, they're essentially equivalent. I'd go with extra-deps just because it's more famiilar.#2020-09-0713:53rickmoynihanexcept :classpath-overrides assumes the lib only introduces a single classpath root, is that right?#2020-09-0713:54rickmoynihanso e.g. it would potentially miss including resources etc#2020-09-0713:55dominicmhttps://github.com/clojure/tools.deps.alpha/blob/f94815dd55bdf5eb30ac8fa075c39e757cbbcca5/src/main/clojure/clojure/tools/deps/alpha.clj#L321 yeah 🙂#2020-09-0713:55dominicmExactly, yeah#2020-09-0714:55borkdudeRe command line args, I have one tool that only has one CLI argument, --opts in which you pass a Clojure map. For that tool I think it made sense and it also simplified command line parsing, possibly also easier to keep it non-breaking. I'm not suggesting that other tools do this, but works for this one particular thing.
clojure -A:carve --opts '{:paths ["src" "test"]}'#2020-09-0723:05sogaiui have a tool that does something similar -- though it doesn't use anything like --opts before the string.
it's nice to not have to decide on command line arguments when you don't know what option and arg names are likely to be used more often.#2020-09-0715:03dominicmI suppose the clj way of doing that is would be clj -X:carve :paths '["src" "test"]'?#2020-09-0715:04borkdudeI guess so, but if we're going to use quotes, why not quote the whole thing anyway#2020-09-0715:05dominicmI mention it only because I think this is what clj is trying to make the standard for cli tools. While I personally think --path src --path test would be the best, standards are better than better.#2020-09-0715:08borkdudeor :paths src,test since comma is whitespace in Clojure, but not in bash?#2020-09-0715:08borkdude$ bb -e "*command-line-args*" -- :paths src,test
(":paths" "src,test")
#2020-09-0715:09dominicmYou're thinking in Clojure 🙂 I'm thinking of bash. Admittedly, my answer was without context. Generally I'd much prefer an api where paths was the & args at the end so I could just do:
clj -X:carve dev-*
#2020-09-0715:09dominicm(which expands to clj -X:carve dev-src dev-resources dev-etc)#2020-09-0715:12borkdudeIf bash would expand that comma separated, then it would work as passing one value.#2020-09-0715:12borkdudeAnyway, it doesn't.#2020-09-0715:14borkdudeYeah, if keywords would separate other values then :jars *.jar :paths src* would work#2020-09-0715:14borkdudeunless your dir starts with a colon :)#2020-09-0715:15borkdudebut I guess using a comma as explicit separator works:
$ bb -e "*command-line-args*" -- :foo t* , :nums 1 2 3
(":foo" "tags" "test" "test-resources" "," ":nums" "1" "2" "3")
#2020-09-0715:21borkdudeThis doesn't work in powershell (no expansion, comma is gone)
PS Microsoft.PowerShell.Core\FileSystem::\\wsl$\Ubuntu-20.04\tmp> cd C:/Temp
PS C:\Temp> bb -e "*command-line-args*" -- :foo t* , :nums 1 2 3
(":foo" "t*" ":nums" "1" "2" "3")#2020-09-0715:22borkdudeFinally, cmd.exe (no expansion, preserves comma)
C:\Temp>bb -e "*command-line-args*" -- :foo t* , :nums 1 2 3
(":foo" "t*" "," ":nums" "1" "2" "3")#2020-09-0715:36dominicmNo idea how cmd.exe handles wildcards. Ultimately there's some platform specific tweaks to match the ergonomics of a platform you're on.#2020-09-0715:37dominicmhttps://superuser.com/questions/1308329/move-folders-using-wildcard#1308364 welp#2020-09-0715:37dominicmhttps://docs.microsoft.com/en-us/cpp/c-language/expanding-wildcard-arguments?view=vs-2019 apparently you have to use a custom routine or something... gosh#2020-09-0715:38borkdudeI guess there's a reason WSL2 is getting popular#2020-09-0715:40dominicmBut also - this is what people on cmd.exe expect. That's fine and while I don't understand it, if that's what makes sense for their OS then fine.#2020-09-0715:42borkdudeI listened to a podcast by someone who is working on the new Terminal app. She said that they couldn't change cmd.exe's behavior because that would be breaking for lots of people. But everyone basically agrees that it sucks.#2020-09-0715:43borkdudeMaking the cmd.exe window smaller will make it faster, because there is less rendering, which is blocking#2020-09-0715:44borkdudeok sorry, got a little bit off topic#2020-09-0717:49lreadGetting back to clojure/clj command line arg compatibility, this is not helpful right now, but I am curious. If we could go back in time would these tools have been better initially named something like clj-alpha1 and clojure-alpha1? This would have drawn attention to their alpha-ness and also allowed a path for introducing breaking changes.#2020-09-0717:51lreadThen again maybe not… this would have implied clojure itself is alpha.#2020-09-0717:51lreadTricky little problem.#2020-09-0717:55borkdudeThe help could have mentioned the word alpha at least somewhere. From and end user's perspective, it's hard to see this is alpha, you'd have to know what .jar it's calling underneath which has alpha in the name. This entire page doesn't mention alpha neither: https://clojure.org/guides/deps_and_cli#2020-09-0717:59seancorfieldI just got a bug report against the new version of clj-new and it's obviously broken but it leads me to a question about t.d.a and migrations: clj-new uses t.d.a to resolve coordinates for templates and it was using t.d.a 0.8.677 and the clojure.tools.deps.alpha.reader namespace with default-deps and read-deps. I updated it to 0.9.782 and switched the code over to pull in the runtime basis... but of course that doesn't exist in the new CLI! Doh! I could revert to 0.8.677 and continue using the old logic, but I was wondering if there was a way in 0.9.782 to get at that same logic so that utilities could run with the basis, if it exists, else fall back to the equivalent of (read-deps (default-deps))? Mostly a question for @alexmiller I guess...#2020-09-0718:15alexmillerThose functions now exist in the tools.deps.alpha namespace#2020-09-0718:16alexmillerOr their equivalents - there were some name changes etc#2020-09-0718:18seancorfieldHmm, it looks like some of the logic for finding deps.edn has moved into the CLI script, and is no longer in t.d.a?#2020-09-0718:18alexmillerNo, should all be there#2020-09-0718:19alexmillerShape is a little different#2020-09-0718:19seancorfieldOK, I'll keep digging...#2020-09-0718:19alexmillerSorry, not at a computer to be precise#2020-09-0718:21alexmillerIt’s just root-deps and user-deps and ./deps.edn#2020-09-0718:21alexmillerRead and merge#2020-09-0718:32seancorfieldfind-edn-maps was what I needed so I think I'm good now...#2020-09-0718:52seancorfieldFor anyone else working on t.d.a-based tooling who wants to leverage 0.9.782 but continue to support older CLI versions, here's what I ended up with for the basis:
(delay (if-let [basis-file (System/getProperty "clojure.basis")]
(-> basis-file
(io/file)
(slurp)
(edn/read-string))
(let [{:keys [root-edn user-edn project-edn]} (deps/find-edn-maps)]
(deps/merge-edns (filterv some? [root-edn user-edn project-edn]))))))
That's to replace the equivalent of (read-deps (default-deps)) from t.d.a 0.8.677 -- it's not identical, but for the purposes of equivalence with 0.8.677 it seems close enough.#2020-09-0802:11cflemingIs there any way with deps to manage source artifacts for non-Clojure dependencies?#2020-09-0802:12cflemingOr rather, how could I specify that I would like them to be downloaded, and how would they be distinguishable from binary jars in the output?#2020-09-0802:25cflemingThis seems to be the answer:
{:deps {junit/junit {:mvn/version "4.13"}
junit/junit$sources {:mvn/version "4.13"}}}#2020-09-0802:30cflemingThey do end up on the classpath, which is unfortunate. I guess they could go in an alias.#2020-09-0803:15cflemingWhen using :local/root, the doc says:
> If the jar has been packaged with a pom.xml file, the pom will be read and used to find transitive deps
Is there a way to prevent this from happening? I would just like it to use the jar I give it and not look for transitive deps.#2020-09-0803:22cflemingLooking at the code, it seems like not. My use case here is trying to use deps to manage the dependencies for Cursive. I need to depend on the IntelliJ framework, so I have a script which downloads and unpacks that, and I’m trying to create a deps.edn which refers to the jars within it. However a bunch of those are third party jars which include poms.#2020-09-0803:22alexmilleryou could use :local/root to a directory which has a deps.edn that includes the jar on the :paths#2020-09-0803:23cflemingOh, I can just put jar paths directly in :paths? That’s even better.#2020-09-0803:23alexmillerkind of a hack, but it works#2020-09-0803:23cflemingI guess that makes total sense, but never occurred to me.#2020-09-0803:23alexmiller:paths are just things that get added to the classpath#2020-09-0803:23alexmilleron the source stuff, tools.deps is about making classpaths so that's what you're going to get#2020-09-0803:24cflemingRight. It’s actually much better for this use case since I don’t need to create a bunch of bogus dep symbols.#2020-09-0803:24cflemingOk. Using $sources works fine, so I’ll support that in Cursive, document how to do it and suggest putting them in an alias.#2020-09-0810:24practicalli@alexmiller I'm confused about using -M with a project rather than a specific alias.
In this example I use the hello-world example from https://clojure.org/guides/deps_and_cli#_writing_a_program
A project deps.edn file was created containing the dependency for clojure.java-time and the source code from that page copied into src/hello.clj
clojure -m hello runs the project and returns the time from running the -main function.
However this gives a warning:
WARNING: When invoking clojure.main, use -M
clojure -M runs a REPL
clojure -M -m hello runs the project and returns the time. But then I ask myself what is the purpose of -M
Creating an alias to run the project seems an interesting idea, as I could also set default arguments. I can only get this to work with :main-opts and not :exec-fn
This version of the alias works when calling clojure -M
:aliases
{:run-project {:main-opts ["-m" "hello"]}}
This version of the alias simply runs the REPL
:aliases
{:run-project {:exec-fn hello]}}
Am I missing something from :exec-fn ?
Edit: yes I am.
The -X option is for running :exec-fn and not the -M function, oops.
So clojure -X:run-project should work... except the hello-world project has an unqualified function and cannot be resolved.
Moving the source code to src/practicalli/hello.clj and calling clojure -X:run-project gives an execution error, (ArityException) as the -main function does not take any arguments (defn -main [] ,,,) . Changing the -main function to (defn -main [& args] ,,,) fixes that and calling clojure -X:run-project works.
From a user point of view, that seems quite a lot to extrapolate from the original warning message when running a project with clojure -m hello#2020-09-0814:49alexmillerThe guide hasn’t been updated for the new stuff yet - I’ll do that with the stable release of it#2020-09-0814:57practicalliGreat. That time frame wont impact me at all. Thanks If you want help with the docs, let me know (although you are the expert here)#2020-09-0811:57practicalliThinking about this further, it does feel that the -X flag would be the most important flag in the long term, especially with :exec-arg allowing default arguments that can be over-written on the command line.
-M seems to be more of a (very useful) convenience function for the specific case of running -main-opts configurations. This does remove the need for command line args and bash quoting, so that is valuable. -M is also useful for those aliases that dont work with -X , eg rebel-readline I could only get working with -M .
As more maintainers adopt the new version of Clojure CLI, there is scope to configure all projects and tools to work with -X and eventually make -M redundant.
Using the -M flag seems to help a little when making the change from Leiningen to CLI tools, as setting the main namespace is a familiar concept.
I am sure I am over-simplifying this and probably missing out on a lot of understanding as of yet.#2020-09-0812:42hugodWould -M not continue to be useful for utilities that you expect to be invokable from the shell? How would you allow passing glob’ed files for example?#2020-09-0812:48practicalliYes, it seems the -M option is valuable when the :main-opts data is more intricate. Interesting to see what can be done with :exec-args verses :main-opts .
I do like :exec-args as a means of using edn over strings.
I dont believe I've have any use cases for using glob'ed files, so cant comment on that without specific examples. I assume something like *.jpg would just be a command line argument after the -X:alias-name and passed to the function.#2020-09-0813:04hugodiuuc, -X expects arguments to be key-path edn-value pairs, so clojure -X:my-alias :x "1 2 3" ends up passing {:x 1} to the exec-fn.#2020-09-0813:18hugodIn general, if you want something that can be be composed with output from tools that don’t produce edn, I think -M is the easier (only?) option.#2020-09-0813:18practicalliAh, so I assume clojure -X:alias :x '["1 2 3"]' is required for :exec-fn , which fits in the concept stucture over strings. Although more specific options seem preferable.
clojure -X:alias :x 1 :y 2 z: 3 .
This is less ambiguous and ties into the general approach I use for Clojure arguments.#2020-09-0813:18practicalliStill lots to learn, so interesting to hear these thoughts. Thanks.#2020-09-0813:19hugodI’m still trying to grasp the implication of the changes too.#2020-09-0814:18alexmiller@U05254DQM I think mostly what you’ve gleaned is correct. -M is for pass through to clojure.main. (And originally that’s the only thing clj did.) Calling with no exec opt is for repl. Right now that’s done with clojure.main but you should consider that an implementation detail. -X is for the new function execution style invocation.#2020-09-0814:20alexmillerWe expect most tools are better off with -X, but totally possible to support both of course#2020-09-0814:23alexmillerFor your own app, not sure yet how this will be taken up. Using -X means you are at least somewhat bound to using clj as your launcher (as the exec stub code is not available anywhere else (yet)). Rich seems to think that’s fine, I’m a little less sure about it. :)#2020-09-0814:25alexmillerMy biggest issue with the -X arg passing is that passing a string path as the v requires wrapping in multiple quotes which kind of ruins - expansion and other glob and auto completion stuff. I’d say that’s not totally finalized#2020-09-0814:43practicalliThanks for confirming Alex. I am far more confident in my understanding of the changes. Apart from updating practicalli/clojure-deps-edn with either -M or -X variants instead of -A, there seems no impact on my development projects.
My notes from a project developer are shared on this gist: https://gist.github.com/jr0cket/ae7dd745eb45870109ace59fe835ce80#2020-09-0814:47practicalliUpdating the Practicalli books with the new approach is pretty easy and can run with the 'new' and 'classic' options for a while.
I've planned screencasts of using Clojure CLI tools for Clojurists Together over the next 2 months. I would rather not re-do them as high quality screencasts are time intensive. I will need to decide when to do these videos, so any thoughts on a release date for the new features would help my own planning (I appreciate any comments on dates are no guarantee).
If a release is not planned until December, I'll opt for some lower quality screencasts and recreate them running up to a release.#2020-09-0814:48alexmillerI’m hoping to get to a new stable release in the next week or two #2020-09-0812:43jeremysHi, I have a couple questions about deps API. There is there a complement function to slurp-deps that also merges the 'user wide' deps.edn and the 'system wide' one ? Also the docs says that resolve-deps and make-classpath are supported. Then the question is: is make-classpath-map to be considered private api ? (and so subject to change?)#2020-09-0814:11alexmillerYou can use find-edn-maps to read and merge-edns function to merge edn maps once they’ve been read#2020-09-0814:12alexmillermake-classpath-map is public but some details of the make-classpath details are going to change, probably this week#2020-09-0814:13alexmiller(Regarding path ordering)#2020-09-0814:16alexmillerThe whole make-classpath area is in a bit of transition#2020-09-0814:25jeremysOk Alex, thanks a lot! I have been foolishly experimenting with the idea of making https://github.com/JeremS/mbt based on tools deps and I ran into those questions today. If I may I have one more concerning tools.gitlibs. Right now it uses jgit v4. Do you plan to use v5?#2020-09-0814:31alexmillerYeah, I’ve spent some time on it but not enough to get it over the line. It also plays into the jsch ssh stuff and look at their newer impl with Apache Mina. #2020-09-0814:31alexmillerJust FYI, I have built a tools.build tool that will hopefully be making its way toward release soon#2020-09-0814:43jeremysOh man... my rationale was that tools.deps begs a tools.build... A proper tools.build is very good!
The jgit question was because the clj-git project uses the v5 making it, I believe, incompatible with tools deps
(breakage between jgit 4 and 5)
Thank you again for the info.#2020-09-0817:24seancorfield@U0FR867U1 If it helps you, take a look at https://github.com/seancorfield/clj-new/blob/develop/src/clj_new/helpers.clj#L60-L71 -- This uses the latest tools.deps.alpha (0.9.782) but still works with both CLI stable and CLI prerelease.#2020-09-0817:25seancorfieldRe: find/read/merge#2020-09-0819:38jeremysThank you @seancorfield the last 2 lines are almost exactly what I need!#2020-09-0815:04borkdudeMaybe aliases should also become fully qualified?#2020-09-0815:04borkdudeThen we can have a global (worldwide) alias registry =)#2020-09-0815:05borkdudeI wasn't trolling (at least, not consciously)#2020-09-0815:05mpenetI didn't mean it that way 🙂#2020-09-0815:07borkdudee.g. want to use seancorfield's aliases -A:org.seancorfield/rebl#2020-09-0815:14borkdudeI guess the problem with that is that aliases can change over time and effectively become a dep. Importing aliases from libs might be an idea, but maybe a bit too much.#2020-09-0816:02alexmilleraliases can be fully qualified now#2020-09-0816:02alexmillerpeople typically don't because more typing, but that's fully supported#2020-09-0908:39rickmoynihanI’ve been doing this for a while with local overrides, e.g. :local/libname will be defined in ~/.clojure/deps.edn for when I have a local checkout of it. I think it works quite well, also to distinguish user wide aliases from project ones.
cc @borkdude#2020-09-0908:41borkdudeyes, doing something something similar#2020-09-0816:04alexmillerone thing we've done a bit of thinking about is - how could I specify a tool to use and have it automatically pulled from github without needing to declare an alias at all#2020-09-0816:04borkdudeyes, that was a bit was I was thinking with this#2020-09-0816:05borkdudebit like lein new does maybe#2020-09-0816:05borkdudeAliases can be fully qualified now, but will it ever be deprecated that they aren't - as with deps#2020-09-0816:05dominicmNeed some aliases procurers#2020-09-0816:06alexmillerprobably not going to fully tackle that for a bit, but I think it would be pretty useful. you need some kind of naming convention that takes you to a gh repo, that can be a deps.edn with deps etc#2020-09-0816:06alexmillerI think just github support would take you pretty far, but yes all that needs to be thought about#2020-09-0816:06alexmillerthere is also some open space I've been working in around discovery with -X#2020-09-0816:06borkdudethis can probably be solved in a tool which you can put in your aliases#2020-09-0816:07alexmillerlike could -X have support for listing the functions available to call#2020-09-0816:07borkdudeI wasn't trolling! geez, what is it with people today ;)#2020-09-0816:07alexmillerhow can you support automatic documentation of a -X compatible function#2020-09-0816:08borkdudemaybe metadata?#2020-09-0816:08borkdude^:x-callable#2020-09-0816:08alexmillerall tbd, but with combination of mapping from alias to tool repo, and support for :ns-default (to find the default tool namespace), and ability to discover the -X functions, and help for each option....#2020-09-0816:09alexmillerthat's an ecosystem#2020-09-0816:09borkdudeyes, the mapping from alias to tool repo was what I was getting at earlier#2020-09-0816:09alexmillerand then if only had a build system that could make use of those functions...#2020-09-0816:10borkdudebuild system?#2020-09-0816:10borkdudetroll#2020-09-0816:10alexmiller:)#2020-09-0816:11dominicmMetadata wouldn't work by itself, you'd have to require every namespace.#2020-09-0816:11alexmilleranyhow, I'm on "vacation", so back to vacating...#2020-09-0816:11borkdudeenjoy!#2020-09-0816:11dominicmYes. Begone!#2020-09-0816:14borkdudeMaybe aliases aren't the right fit for name -> github repo and some other concept/option should be invented for that.#2020-09-0817:27seancorfieldSort of related to this line of thinking: clj-new supports templates on GitHub via naming conventions, so maybe have a look at the readme for the usage and see if it is close to what you are thinking @borkdude?#2020-09-0817:31borkdude@seancorfield You mean:
clj -A:new -M:new ...
?#2020-09-0817:33borkdudeor do you have some short form where you only mention the org + repo name, with automatically resolving the newest sha?#2020-09-0817:43seancorfieldCurrently, you need the full URL but that's really only to detect what type of template you're using. If you know it's GH, you could take just the org/repo-name and build the rest. And my dot-clojure file has an example of using tools.gitlib to figure out the HEAD of master SHA.#2020-09-0817:44seancorfield@borkdude See https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L174-L177#2020-09-0900:24cflemingJust double checking - there is no way to get the deps of a subproject specified via :local/root including deps from an alias, correct? i.e. I cannot specify an alias together with :local/root in some way?#2020-09-0900:26seancorfieldNope.#2020-09-0900:26seancorfieldDependencies are opaque in that respect.#2020-09-0900:28seancorfieldAlthough... I guess you could read the basis (in the prerelease CLI) and figure out the actual :local/root path and see if it has deps.edn and then use t.d.a to read that and then inspect its aliases... 👀#2020-09-0900:29seancorfieldBut a :local/root could point to a JAR, or a directory with pom.xml. Or project.clj 🙂#2020-09-0900:32cflemingRight.#2020-09-0900:33cflemingThis is just specifying deps in a deps.edn file, so I can’t do anything funky or programmatic.#2020-09-0901:13hugodWhat is the rationale for deprecating unqualified dependency symbols in :deps? (couldn’t see one in the release notes)#2020-09-0901:17seancorfield@hugod It's mentioned in this post https://insideclojure.org/2020/07/28/clj-exec/ near the end.#2020-09-0901:18seancorfield(IOW it was deprecated in an earlier prerelease)#2020-09-0901:20hugodThanks - I missed that release 🙂#2020-09-0901:25seancorfieldI spent the first week of August, scurrying around, fixing all our unqualified lib names at work, and all my open source projects! 🙂#2020-09-0902:31cflemingIs there any way to substitute values into a deps file that I’m not aware of, e.g. from env vars or properties? In my case I would like to be able to substitute a global version in several deps files. I’m aware of default-deps, but I’d need to be able to substitute it in a :local/root path.#2020-09-0902:33seancorfieldNope. No substitutions.#2020-09-0902:39seancorfield(I'm just a bundle of "nope" today, aren't I?)#2020-09-0902:40cflemingHaha#2020-09-0902:40cflemingNo worries, that was what I expected, just makes my life a little more tedious 🙂#2020-09-0913:19hugodFwiw, having played with the pre-release, I still think the following CLI switches would be simpler, as they separate the selection of aliases from what to execute, and would simplify the doc. A given set of aliases can be used for main, exec or repl, so it makes it easier to compose a command line from other tools if the passing of options is distinct from the selection of what to invoke.
clj [clj-opt*] [-A:aliases] [init-opt*]
clojure [clj-opt*] [-A:aliases] -x [a/fn] [kpath v]*
clojure [clj-opt*] [-A:aliases] [init-opt*] [main-opt] [arg*]
clojure [clj-opt*] -P [other exec opts]
or it could be
clojure [clj-opt*] [-A:aliases] -M [init-opt*] [main-opt] [arg*]
If that is really required, though I confess I don’t see the rationale for having to specify the -M given that you’re passing -m or have a :main-opts key.
The (at least my perceived) lack of usage of the current -T, `-R`, `-C`, and -O options supports the above.#2020-09-0913:21alexmillerSorry, not doing that#2020-09-0913:29hugodThat’s clear 🙂#2020-09-0916:15seancorfieldAlso, I think there's more usage of -R out there than you suspect @hugod#2020-09-0916:22hugodI’m sure I don’t have a good perspective on that, but don’t think it really changes my view that just using -A would be conceptually simpler.#2020-09-0916:22hugodGiven the cli changes seem fixed, I’ll just go and update my nascent build tool invoker project to cope with multiple versions of the cli.#2020-09-0916:28seancorfieldI still think that a) given the expansion of -M to include more than just :main-opts b) the overlap between -A and -M (both run :main-opts now) c) the current uses of -R (doesn't run :main-opts but does "most" of what -A does -- based on resolve args being the most common; it would make more sense for -R to be kept and expanded as the "REPL" usage (and quietly undocumenting -A). So we'd have -R (REPL), -M (main), -X (exec) and they all three respect all arg types in deps.edn and there would be no overlap on :main-opts or :exec-fn.#2020-09-0917:20hugodI do find it unfortunate that the deps.edn isn’t authoritative. By this I mean that whatever you specify in deps.edn you still have to know whether to invoke it with -M or -X.#2020-09-0917:29seancorfieldSee https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L15-L18 -- you can support both 🙂#2020-09-0917:30seancorfieldThat works with -A today and either -M or -X tomorrow!#2020-09-0917:36hugodI don’t think that is quite what I meant. As a deps.edn writer I would like to provide an abstract, say test, alias for doing something, and have the user be able to invoke that independent of how I chose to implement that alias. Writing it both ways seems like duplication.#2020-09-0918:55alexmillerwell the arg structures are different, not sure how that could work#2020-09-0919:45seancorfieldExactly, as a tool writer you need to specifically support -M style invocation and/or -X style invocation as two separate functions in your code: -main taking & args which are strings and some-exec-fn taking a single hash map as its argument.#2020-09-1013:04hugodmain obviously takes strings and an exec-fn wants an edn value, but that doesn’t need to dictate the format of arguments passed on the cli. The cli could print edn to strings before passing it as a main argument, or read literal strings to edn strings before passing to an exec-fn. As a strawman, with intentionally hideous names, consider this:
i) default to passing args as literal string values, or add a --string switch to explicitly specify this
ii) add an --edn switch, that causes the next argument to be read as EDN and passed as a value to an exec-fn or as a string to a main.
iii) add a --patch switch that takes an edn path vector and an edn value and patches the default value. if the value is prefixed with --string treat it as a string value rather than EDN. Allow main to have default arguments specified, in the same way as exec-fn.
iv) add a --rest switch that collects the remaining arguments into an EDN vector. --rest-strings could build an EDN vector from string arguments rather than edn values. This could be used to pass glob values, for example.
v) allow an exec-fn to take multiple arguments.
As described this is unwieldy, but if there were volition, I’m sure it could be improved upon. The main idea would be to divorce the specification of the format of the arguments passed on the command line from how they are passed to the implementing function.#2020-09-0919:08jeremysHi, I have been working on a https://github.com/JeremS/mbt based on tools.deps for some time. It has a mix of functionality inspired from several other projects and it's at a stage where I use it to version, build and deploy my projects. Maybe someone other than me might find it useful! Cheers#2020-09-0919:37seancorfield@alexmiller It looks like clojure -X:deps mvn-pom now adds the following dependencies to all pom.xml files:
<dependency>
<groupId>org.clojure</groupId>
<artifactId>tools.deps.alpha</artifactId>
<version>0.9.782</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-nop</artifactId>
<version>1.7.25</version>
</dependency>#2020-09-0919:39seancorfieldThat's going to cause a lot of projects to accidentally end up depending on tools.deps.alpha (and the slf4j-nop dependency is likely to be unwelcome in many cases I suspect).#2020-09-0919:40seancorfieldThese deps are in a freshly-generated pom.xml file -- and they are also added to any existing pom.xml, potentially overwriting the dependency for any projects that use a different version of t.d.a#2020-09-0919:43alexmillerah, interesting#2020-09-0919:43alexmillerthx, will fix#2020-09-0919:59seancorfieldHow will you fix it? For example, my clj-new project does depend on t.d.a (and has a dependency on slf4j-nop because of that) -- but I know that's an outlier -- so you'd need to also not remove any existing dependency on those?#2020-09-0920:00alexmillerI'll fix it by calculating the basis, not using the injected basis#2020-09-0920:01alexmillerbasically what it did before#2020-09-0920:02seancorfieldWon't that break anyone currently using clojure -A:some:aliases -Spom ?#2020-09-0920:11alexmillerit doesn't have to - the injected basis includes those argmaps#2020-09-0920:11alexmillerthe calculated basis can use the project deps.edn + the injected basis argmaps#2020-09-0920:25seancorfieldRe: basis -- so -A and -X are distinguishable via the basis? Or am I misunderstanding?#2020-09-0920:26alexmilleroh, I see what you're asking. well let me think about it more#2020-09-0920:12alexmilleranother option would be to take those additional aliases as args to the -X pom function#2020-09-0920:12alexmillerthat would be an all new thing though#2020-09-0920:13alexmillervery similar thing exists in the datomic ion-dev tool which I was just helping with a bit :)#2020-09-0920:14alexmillerI am planning to reinstate the -Spom as well so looking at that right now too#2020-09-0920:26seancorfieldIf it's just going to be identical to -X:deps mvn-pom I wouldn't have thought it was worth the effort?#2020-09-0920:27alexmilleryou were the one here arguing it was breaking people last week... :)#2020-09-0920:31seancorfieldHahaha... yeah, but you didn't seem very sympathetic to that argument, and I'm not sure having duplicate ways to do the exact same thing is better 🙂#2020-09-0920:14alexmillerI had some thoughts on the -Stree thing you ran into though - I think there are several better ways to detect changing deps tree rather than scraping the -Stree stdout. not sure what your requirements on that stuff are but might be useful to talk about#2020-09-0920:18alexmillerfrom a clj pov, we're writing edn files (.libs in particular) to the cache dir (can see location with -Sdescribe), so you could be slurping and diffing Clojure maps rather than comparing strings#2020-09-0920:27seancorfieldTrue, but a) how can you easily get the correct .libs file from a given invocation without scraping the output and b) that gives no sense of depth of transitivity, which is specifically what I rely on in -X:deps tree#2020-09-0920:28seancorfield"Because reasons"... we go one level down in the full tree (so we consider top-level deps and the first level of transitive deps, but we don't care about lower levels).#2020-09-0920:30seancorfieldWe've found that we can safely ignore the 2nd-order transitive deps and if we have a problematic 1st-order transitive dep, we just promote it to the top-level (and then its transitive deps are promoted to 1st-order transitive deps and we start to track them separately).#2020-09-0920:30alexmillerwell the transitivity stuff is all broken in the -Stree output#2020-09-0920:31alexmillerbut maybe you're just not encountering the cases where it's bad (mostly with exclusions)#2020-09-0920:31seancorfieldWe don't use exclusions, for the most part.#2020-09-0920:32seancorfieldThis is our only exclusion:
com.walmartlabs/lacinia {:mvn/version "0.35.0"
:exclusions [clojure-future-spec/clojure-future-spec]}#2020-09-0920:32alexmillerthe main thing is, the code never actually builds the real expansion tree as a tree. -Stree is kind of faking from the libs map#2020-09-0920:33seancorfieldWe decided that managing conflicts via :exclusions was a bit of a nightmare so we control conflicts by promoting transitive deps to the top level.#2020-09-0920:33seancorfieldRight, but I'd have to write that "faking" code myself, whereas -X:deps tree already does a "good enough" job.#2020-09-0920:34alexmillerwell, it exists as a function you could call (the one -Stree calls) :)#2020-09-0920:34alexmillerbut that stuff is due for some significant changes soon as I fix up some of this stuff#2020-09-0920:34seancorfieldBesides, I've had this shell script running with -Stree for ages and don't really want to rewrite it as a Clojure program right now. Maybe when I'm bored and want to go shave some more yaks...#2020-09-0920:35alexmillerok, just letting you know that I'm not considering stdout changes from -Stree (or its replacement) to be breaking :)#2020-09-0920:36alexmillerbut I am planning to add some more api fns to get tree-like deps data#2020-09-0920:19alexmillerand from a tools.deps pov, you could probably make something that did this via the api instead, but not sure if that's worth the effort#2020-09-0920:21alexmillerI really consider -Stree / -X:deps tree to be human consumer tools and ones likely to hopefully improve over time, changing format. I don't consider the current particular output to be something a script should rely on.#2020-09-1000:55didibusI'm excited to try the new clj exec, at first glance, its giving me some powershell vibes... which I'm not sure about 😛#2020-09-1001:00didibusOne question, in the guide it says: clj -A:deps -X:install :jar '"/path/to.jar"', couldn't you include the deps in the :install alias itself? And thus you could then omit the -A:deps and just do: clj -X:install :jar ...#2020-09-1001:14seancorfieldIt says clj -X:deps mvn-install :jar '"/path/to.jar"'#2020-09-1001:14seancorfieldAre you looking at the old (stable) guide @didibus?#2020-09-1001:14seancorfieldThe prerelease is https://clojure.org/reference/deps_and_cli_prerelease#2020-09-1001:15seancorfieldBut, yes, in general you can combine the exec stuff and the dependencies.#2020-09-1001:16seancorfieldI have some examples of that here https://github.com/seancorfield/clj-new#clojure--x-usage#2020-09-1001:38didibusI probably was, it was from the first blog post about it on inside clojure#2020-09-1001:39didibusAh yes, the latest blog post says: "so you can supply the deps and the function to execute together" never mind then#2020-09-1001:42didibusSo that's awesome, now all we need is a convention to define alias in some file that you put in a git repo, and something like: clj -X:deps install-alias some-git-repo and a corresponding clj -X:deps remove-alias some-git-repo and we have ourselves a little NPM of some sort.#2020-09-1001:43didibusWell, it doesn't need to live inside -X:deps, but it would be an easier way to bootsrap it#2020-09-1001:45didibusYou could even do something like: clj -X:deps install-alias :git some-git-repo :symlink true and it could create a symlink in your user /bin to: clj -X:your-alias#2020-09-1001:45didibusSorry, none of this has to be included in deps, though I think it could be a good idea#2020-09-1001:46didibusThen clj could be used to install Clojure command line apps on your machine as well.#2020-09-1001:47didibusMight need some more hammock time#2020-09-1114:58souenzzoI'm using clojure 1.10.1.645 on linux
When I do clojure -e "(compile 'app.main)" it results in java .... -e '(compile '\''app.main)' wich works
When I create a alias :aot {:main-opts ["-e" "(compile 'app.main)"]}
It results in java commnad like -e '(compile' ''\''conduit.server)' which will do a EOF error, as it will try to eval (compile
Is it a know issue? I think that I already used it before (it looks like a regression for me)
PS: I'm using bash -x /usr/bin/clojure ... to know which "final java command" clojure is calling#2020-09-1115:00alexmillernothing has changed in this area afaik#2020-09-1115:01alexmillerthings in :main-opts go into a file, and then back out of a file which causes problems due to word-splitting in bash#2020-09-1115:01alexmillerso if you replace the space after compile with a , that will help#2020-09-1115:01alexmillerthis is a known issue (but it has always been a known issue :)#2020-09-1115:30souenzzoShould I try to PR?
I worked with bash for some years 🙂#2020-09-1115:30alexmillerthere is a ticket and about 4 different patches on there already :)#2020-09-1115:31alexmillerlet me find it#2020-09-1115:33alexmillerhttps://clojure.atlassian.net/browse/TDEPS-56#2020-09-1115:35alexmillersimilar but different issues on windows#2020-09-1117:01lreadHere’s related issue around Windows https://clojure.atlassian.net/jira/software/c/projects/TDEPS/issues/TDEPS-133#2020-09-1117:17seancorfieldI just routinely use , instead of whitespace when writing stuff in deps.edn and even on the command-line for clojure 🙂#2020-09-1117:20lreadaka the “corfield comma”!#2020-09-1117:23seancorfieldI can think of worse things to be known for 🙂#2020-09-1117:51souenzzoNow thinking about Windows.
I can't implement a bash-friendly solution, once it may be dificult to implement in "cmd", once it may need to "interpret" that file as bash.
How is going that idea of do like "deps.exe" and use a graal-binary to replace bash/bat?#2020-09-1118:22seancorfieldOr use WSL2 for Clojure development? 🙂#2020-09-1118:49lreadIf you want to work on plain old Windows and don’t need to stay with official Clojure tools, you might consider trying https://github.com/borkdude/deps.clj. Using its deps.exe in place of clojujre.exe made sense for me when testing my project under Windows on GitHub Actions.#2020-09-1117:18alexmillernow of course, you can just use -X in many cases instead :)#2020-09-1117:22seancorfieldAlthough you can't use -X to do a simple compile because it requires a symbol and -X passes a hash map 🙂#2020-09-1117:29dominicmI reckon we'll see a pack of map versions of core functions.#2020-09-1117:48vlaaadAll you need is a single invoke :
(defn invoke [{:keys [fn args]}]
(apply fn args))
(invoke {:fn inc :args [1]}) => 2
#2020-09-1117:29dominicmOr maybe the work on kwargs, will include positional arguments too?#2020-09-1118:49alexmillerA new prerelease version of clj is now available (1.10.1.681):
• Reinstate -R, -C, and -Spom from last week's prerelease (but still deprecated)
• TDEPS-155 - Some error message improvements for various bad coordinate cases#2020-09-1118:49alexmillerBased on feedback from last week, added some things back to ease the migration.#2020-09-1118:52alexmillerFor the moment -Spom will not warn. The -X:deps mvn-pom version was really missing support for modifications to the classpath previously pulled from clj invocation itself. I've added that as an arg but needs more thinking.#2020-09-1118:54alexmiller@seancorfield I've also standardized all the deprecation warnings to be stderr, if you wanted to pipe redirect them to /dev/null and ignore that's at least possible now. also I fixed the exec.jar issue in the installation.#2020-09-1119:24seancorfieldThanks @alexmiller -- just to confirm, is -R exactly the same as before? It hasn't expanded to support other types of arg maps like -A?#2020-09-1119:38alexmilleryes, just re-added, not going to do the expanded thing#2020-09-1120:56markbastianI had an uberjar alias using depstar that was configured with main opts of:
["-m" "hf.depstar.uberjar"
"target/myproject-with-dependencies.jar"
"--compile" "*" ;Note - the * seems to be needed for older versions of the args, but not now.
"--verbose"
"-m" "myproject.main"]
It was using the RELEASE version of depstar. I noticed that now instead of putting the jar in target/myproject-with-dependencies.jar it creates a jar named (Note: Do not ".
Is this the right way to build the standalone jar with aot compilation? It looks like --compile will do transitive compilation from your entry point. Is there a way to specify aoting the whole project? I saw that you can do :aot true. If I did that would it go in the main args or next to the :main-args key in the deps file?#2020-09-1121:05markbastianActually, looks like :aot is part of the -X functionality. I missed that.#2020-09-1121:21seancorfield@markbastian There's a #depstar channel now for dedicated help...#2020-09-1121:23markbastianThanks, I should have thought of that.#2020-09-1202:32hugodWith cli 673, and 681
clojure -Stree => Option changed, use: clj -X:deps tree
clojure -X:deps tree => Unqualified function can't be resolved: tree#2020-09-1203:15seancorfield@hugod It works fine for me (aside from showing tools.deps.alpha as a dependency and everything it depends on -- which was the problem with -X:deps mvn-pom in the last release).#2020-09-1203:15seancorfieldDo you perhaps have your own :deps alias, shadowing the system installed one?#2020-09-1210:56hugodThat was it thanks. Should have thought of that. Might be nice to have a warning if you define your own :deps alias.#2020-09-1203:16seancorfield(! 993)-> clojure -Sdescribe
{:version "1.10.1.681"
:config-files ["/usr/local/Cellar/
^ @alexmiller FYI#2020-09-1203:29sogaiuhere i get:
$ clj -X:deps tree
Cloning:
Checking out: at e160f184f051f120014244679831a9bccb37c9de
Unqualified function can't be resolved: tree
$ clj -h | grep 681
Version: 1.10.1.681
#2020-09-1203:31sogaiulooks like i've got the following in my ~/.clojure/deps.edn:
:deps
{
:extra-deps {org.clojure/tools.deps.alpha
{:git/url ""
:sha "e160f184f051f120014244679831a9bccb37c9de"}}
}#2020-09-1203:33sogaiu...and with -Srepro i don't get that message#2020-09-1203:57seancorfieldYup, bad idea. You should only have that under an alias.#2020-09-1203:58seancorfieldYou're essentially breaking the :deps alias in the system installed deps.edn...#2020-09-1203:58seancorfield@sogaiu ^#2020-09-1205:38dominicmMaybe the system deps should have a namespace?#2020-09-1206:05seancorfieldOr maybe system aliases should be "reserved" so you can't overwrite them?#2020-09-1206:18sogaiu@seancorfield yes, thanks. i think it was a left-over from here: https://github.com/seancorfield/dot-clojure/blob/d4aba12ec24834200b70ee0c06a30fb0538bddc0/deps.edn#L167 - my bad for not keeping up-to-date#2020-09-1206:19seancorfieldYeah, I changed that when I realized that it conflicted with the system one 😞#2020-09-1206:19seancorfieldNow it's this: https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L164#2020-09-1206:20seancorfield(and that's an outdated branch -- the latest versions of add-lib require builds from source because they include Java code)#2020-09-1206:20sogaiuah thanks!#2020-09-1206:21seancorfieldI update that file a lot -- so it's a good idea to keep git pulling it regularly 🙂#2020-09-1206:22seancorfieldI'm adding -X compatible stuff now as well, for folks on the latest prerelease of the CLI: https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L15-L18#2020-09-1206:22seancorfieldBoth -M:new and -X:new will work.#2020-09-1206:23seancorfieldclojure -M:new lib myname/myproject or clojure -X:new :name myname/myproject (since :template is defaulted to lib).#2020-09-1313:35dominicmI suppose that the new -X stuff can't express regex directly, as it's EDN based. I only mention it because it's one of a handful of types that couldn't be passed to a function directly, and a wrapping string-based function would be needed for use from the CLI.#2020-09-1313:49alexmillerYep#2020-09-1317:35seancorfieldYup, that's what I did for depstar's --exclude arg, but the command-line version is also a plain string so both logic paths have to turn it into a pattern.#2020-09-1318:26borkdudesame with EDN config, I also use strings as regex in clj-kondo's config in places#2020-09-1318:32kingcodeI’m wondering about this strange syntax in <install-dir>/deps.edn:
{…
{:aliases ...}
R
...
}
So it looks like an embedded deps map is used as a key pointing at the -R option? How does this work? I couldn’t find this documented anywhere on the clojure site’s Deps and CLI reference. (Using clojure 1.10.1.590)#2020-09-1320:32seancorfield@kingcode I don't see that in any of the versions I have installed -- is it possible you've accidentally edited it?#2020-09-1320:35seancorfieldHere's what's in my system install deps.edn for that version:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "720117131c113236372139263d225f41423b31334544"}, :content ("[email protected]")}#2020-09-1320:40kingcode@seancorfield Sorry, I meant my config dir, not the install.. I think maybe installing rebl recently caused a bad edit - here is the effective content of .clojure/deps.edn:
{ ;...
{:aliases
{:rebl ;; for JDK 11+
{:extra-deps {com.cognitect/rebl {:mvn/version "0.9.241"}
org.openjfx/javafx-fxml {:mvn/version "15-ea+6"}
org.openjfx/javafx-controls {:mvn/version "15-ea+6"}
org.openjfx/javafx-swing {:mvn/version "15-ea+6"}
org.openjfx/javafx-base {:mvn/version "15-ea+6"}
org.openjfx/javafx-web {:mvn/version "15-ea+6"}}
:main-opts ["-m" "cognitect.rebl"]}
:rebl-jdk8 ;; for JDK 8
{:extra-deps {com.cognitect/rebl {:mvn/version "0.9.241"}}
:main-opts ["-m" "cognitect.rebl"]}}}
R
;;....
}#2020-09-1320:41seancorfieldThis is definitely not legal syntax#2020-09-1320:41alexmillerAgreed :)#2020-09-1320:42kingcodeOK…<sigh-of-relief> thx for your comments. I probably should remove the R and make the embedded map the entire effective content then?#2020-09-1320:44seancorfieldHard to say. My ~/.clojure/deps.edn looks like this: https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#2020-09-1320:45seancorfieldBut :aliases should be a top-level key in the overall map.#2020-09-1320:46kingcodeThanks @seancornfield - that makes sense 🙂#2020-09-1321:19dpsuttonAliases always go in a map with the other aliases. That example is correct. And your deps.edn locally will largely look the same yes?#2020-09-1321:28kingcodewoops @dpsutton and @alexmiller Sorry indeed my mistake. The Rebl distro alias example works fine 🙂 Too much staring.#2020-09-1409:59kirill.salykinUPD: nvm, wrongly specified the maven repo, sorry
---
Hi
I am trying to install dependency from the s3 but facing the issue:
Error building classpath. Failed to read artifact descriptor for com.amazon.redshift:redshift-jdbc42:jar:1.2.47
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.amazon.redshift:redshift-jdbc42:jar:1.2.47
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:255)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
(Following this doc https://docs.aws.amazon.com/redshift/latest/mgmt/configure-jdbc-connection.html#configure-jdbc-connection-with-maven)
{:paths ["src" "resources"]
:mvn/repos {"redshift" ""}
:deps{
com.amazon.redshift/redshift-jdbc42 {:mvn/version "1.2.47"}
...
}}
Please help what I am doing wrong?#2020-09-1411:57practicalli@alexmiller Will the changes in the pre-release be released as version 1.10.2, or something similar such as 1.10.2-alpha or 1.10.2-RC ? This would make it very easy to document usage of the tooling, as an easy distinction can be made between 1.10.2 and pre-1.10.2 versions. Thank you#2020-09-1412:34alexmillerNo, probably not#2020-09-1412:40practicalliAh, so I'll just drop the older content then and add hints to require the latest version is running. If issues arise the the hint will be first to update the version of CLI tools. Thanks#2020-09-1417:44unbalancedstruggling with a proxy just trying to get clj -r going#2020-09-1417:45unbalancedI've tried setting the settings in ~/.m2/settings.xml and running
clj -Dhttp.proxyHost=<host> -Dhttp.proxyPort=<port> -Dhttps.proxyHost=<host> -Dhttps.proxyPort=<port> -r
#2020-09-1417:45unbalancedany suggestions?#2020-09-1418:08borkdude@goomba You need to pass Java opts using -J-D#2020-09-1418:14unbalancedahhhhh#2020-09-1418:14unbalanced🙏#2020-09-1517:43Aaron CummingsDoes there exist a tool like Mr. Anderson (dependency shading) for tools-deps?#2020-09-1517:43Aaron CummingsDoes there exist a tool like Mr. Anderson (dependency shading) for tools-deps?#2020-09-1517:52andy.fingerhutI haven't actually used Mr. Anderson, but am aware of it and why someone might want to use it. Is Mr. Anderson in any way currently tied to Leiningen?#2020-09-1518:08Aaron CummingsIt is a leiningen plugin, but it looks like it could be used directly. But I'm having trouble describing what I'm trying to accomplish, which maybe means I'm reaching for the wrong thing. I'll need to think about this some more.#2020-09-1518:28andy.fingerhutI have never done this before, but the Mr. Anderson project README has a brief note and a pointer to this repository, which seems to use Mr. Anderson in a way that doesn't require Leiningen: https://github.com/Olical/conjure-deps#2020-09-1523:07sogaiuit does use it as a library there#2020-09-1600:41Aaron CummingsI'll take a look at that, thanks.#2020-09-1520:36rbergerI am wondering why I’m getting
shadow-cljs - starting via "clojure"
Cloning:
(Full stacktrace at https://gist.github.com/rberger/34d72e06736ed7cb449c88030c9815ba)
This happens if I have the following in my .ssh/config:
Host
User git
Port 22
HostName
IdentityFile ~/.ssh/github.rsa
AddKeysToAgent yes
TCPKeepAlive yes
IdentitiesOnly yes
Host *
User myname
Port 22
IdentityFile ~/.ssh/id_rsa
AddKeysToAgent yes
TCPKeepAlive yes
IdentitiesOnly yes
All my regular git activity works fine with this, and it seems I need this for macOS automatic key loading to work
If I remove the Host * entry the clojure tools git loading works fine.
I presume this is a jgit issue but I’m wondering if there is a way to make this work as I’m expecting or if I’m expecting wrong.
The Maven issues talk about setting to useNativeGit=true but I don’t know if there is a way to do it with the clojure tools
I had this problem once before and removed the offending config stuff, but I would like to figure out how to make this work with this or equivalent ssh config if possible.#2020-09-1520:50seancorfieldIdentityFile is the problem I believe, based on discussions I've seen here...#2020-09-1520:51seancorfieldBut @alexmiller can give you a more authoritative answer...#2020-09-1520:53alexmillerthat's definitely where I'd start (by removing that)#2020-09-1520:56alexmillersince it looks like you are using a different github key, I'm not sure of the right combination of things to make it work. you might need to add that key with ssh-add#2020-09-1611:52practicallihttps://github.com/practicalli/clojure-deps-edn/tree/qualified-alias-keywords-and-new-flags is a refactor of the project wide aliases in light of the changes coming in Clojure CLI tools (version 1.10.1.681 onwards). This will go live when the Clojure CLI changes go live.
The aliases are using the -X flag for executing functions and providing default hash-map arguments (where the library supports this) and using -M flag for clojure-main (replacing -A and -R). Where chained aliases in a command have multiple main namespaces, the alias that should have its main run is added last in the chain.
Alias namespaces have been upgraded to be qualified names, further defining their purpose and making them easy to understand without looking at the details (at least that is the intent).
https://gist.github.com/jr0cket/ae7dd745eb45870109ace59fe835ce80 contains notes to be added to the Practicalli Clojure book on using the Clojure CLI tools with the recent changes. Any feedback on this is welcome, especially on how clear an accurate this is.
Thank you.#2020-09-1613:52alexmiller-P can be used with -X too#2020-09-1613:55alexmillerre "The order in which repositories are consulted is not guaranteed", recently I changed this to actually guarantee that the first two repos consulted are central and clojars (no guarantee after that). central and clojars are listed in the root deps.edn so do not need to be added to deps.edn or settings.xml unless you add mirrors etc.#2020-09-1613:56alexmiller"`business-area**" looks like bad markup#2020-09-1613:58alexmillerthe automatically included "central" and "clojars" repos can be removed from consideration by nil'ing them in deps.edn {:mvn/repos {"central" nil}}#2020-09-1617:08seancorfieldcurl -O
chmod +x linux-install-1.10.1.681.sh
sudo ./linux-install-1.10.1.561.sh
Version is not correct in that last line.#2020-09-1617:09seancorfieldRun project with or without an alias:
`clojure -M:alias -m domain.app-name`
`clojure -M -m domain.app-name
`
Looks like you have extra backticks in that code block.#2020-09-1617:10seancorfield:project/outdated-mvn - check for newer dependencies (maven only)
The other two lines above this have the alias as code (with backticks) but this third one does not.#2020-09-1617:12seancorfieldhttps://cljdoc.org/d/seancorfield/next.jdbc/1.1.588/doc/getting-started#create--populate-a-database
I think it would be better to use CURRENT instead of a specific version there, unless you plan to ensure that you update the README any time you update the next.jdbc version used in any of your aliases?#2020-09-1617:12seancorfieldclojure -M:inspect/portal-cli ClojureScript node.js REPL with Portal dependency
That should be :inspect/portal-node#2020-09-1617:15seancorfieldAdd Reveal as tap> source
I thought Reveal did that automatically? I don't recall having to do this manually when I've used Reveal. Depends on how you start it. If you start it with vlaaad.reveal/repl the add-tap is automatic. Perhaps make it clearer that this section only applies if you have started the REPL some other way than via Reveal itself? Or maybe it was just my misreading of what's being discussed at this point in the README?#2020-09-1617:15seancorfield"DEPRICATED" => "DEPRECATED"#2020-09-1617:15seancorfieldI found several other aliases further down the README that are not highlighted as code with backticks.#2020-09-1617:22seancorfieldhttps://github.com/practicalli/clojure-deps-edn/blob/qualified-alias-keywords-and-new-flags/deps.edn#L444 -- midje-runner is not unqualified in the alias -- but should be midje-runner/midje-runner (since unqualified lib names just double up in the group and artifact slot), otherwise you might cause a clash with any midje-runner lib that actually ends up with the midje group ID.#2020-09-1617:24seancorfieldhttps://github.com/practicalli/clojure-deps-edn/blob/qualified-alias-keywords-and-new-flags/deps.edn#L575-L590 -- the documented -X entry points are hf.depstar/jar and hf.depstar/uberjar (the ones you have still exist for backward compatibility but are not documented).#2020-09-1618:52practicalliThank you Alex, Sean, most helpful and much appreciated.#2020-09-1621:02lreadVery cool @U05254DQM! Not sure if anyone mentioned what looks like a typo:
* deps = `:deps`, `:extra-deps`, `replace-paths`
* path = `:path`, `:extra-paths`, `replace-deps
`
Last items seem reversed?#2020-09-1621:59practicalliGood catch. @UE21H2HHD , I'll fix that copy past error, thanks.#2020-09-1623:30seancorfieldPosting here because it there's a larger audience who might have more insight into Maven and its shade plugin behavior: The next version of depstar will preserve license/copyright files coming from dependencies by concatenating them (but omitting exact duplicates). As far as I can tell from inspection of several "uber" JAR files I found in the wild, this is the behavior of Maven with the shade plugin, with the additional license resource transformer (which omits duplicates). I couldn't find anything in Maven's documentation about this behavior, and looking at the source of the shade plugin didn't help much. If anyone here has deep insight into that aspect of Maven and can confirm its actual behavior, that would be very helpful (otherwise I'll just run with the depstar behavior I described).#2020-09-1623:39alexmillerI've used the shade stuff several times and I'm not sure it is anything mandated by maven either way, you're in making stuff up territory :)#2020-09-1623:40alexmillerI actually have come to think that the "right" way to do this is store things like that in lib-specific sub dirs under META-INF#2020-09-1623:40alexmillerI don't think concatenating is wrong though#2020-09-1623:55seancorfieldYeah, my inspection of various "uber" JAR files seemed to confirm that if maven shade finds META-INF/LICENSE or META-INF/LICENSE.txt in multiple dependencies, it simply concatenates them together in the resulting JAR file. My reading of the Apache license resource transformer seemed to confirm that it attempts to avoid duplicating the text of any license.#2020-09-1623:55seancorfieldBut this is definitely all "undocumented territory" as far as I can tell.#2020-09-1701:03pmonksI wonder if there are legal implications when concatenating license texts?#2020-09-1701:07alexmillerseems dubious to me :)#2020-09-1714:13markbastianWhen I run clj -Spom with an existing pom that already has a <build> tag the generated <build> tag is then placed inside of the existing rather than merging the contents or just adding the <sourceDirectory>. Is this a bug or am I just doing it wrong?
To be clear, it does this:
<-- Contents above elided -->
</plugins>
<build>
<sourceDirectory>src/main/clojure</sourceDirectory>
</build>
</build>
</project>#2020-09-1714:17alexmillerwhat version are you on?#2020-09-1714:17alexmillerthere have been bugs and fixes on this#2020-09-1714:17alexmillerclj -Sdescribe#2020-09-1714:23markbastian"1.10.1.561"#2020-09-1714:31alexmilleryeah, that's the newest stable but the fix is not included in a newer stable version yet (it's fixed in the prereleases but you'll get a bunch of other changes with that)#2020-09-1714:33alexmillerso you could move to a prerelease (1.10.1.681 is newest) or you could fall back to a prior stable version - 1.10.1.510 or earlier#2020-09-1714:33alexmillerthat version was actually also broken but only for -Spom generating a new pom#2020-09-1714:34alexmilleror you can wait till the next stable version, hopefully coming within the next week#2020-09-1714:34borkdudeor generate the pom with the newest unstable and then move back to the newest stable :)#2020-09-1714:36borkdudeit's pretty easy to do this if you download the clojure shell script for a given release#2020-09-1714:38alexmillerI prefer "prerelease" to "unstable" :)#2020-09-1714:45borkdude@markbastian if you're on mac, this is how you can switch versions easily:
brew install clojure/tools/
#2020-09-1714:46borkdudeor if you're using linuxbrew, same#2020-09-1715:10markbastianCool, thanks for the help. Waiting for next week is great. I don't need to update the pom much.#2020-09-1716:42borkdudeWould it be useful if clj-kondo did some checking on deps.edn (statically) for common mistakes?
Feel free to post ideas here: https://github.com/borkdude/clj-kondo/issues/945#2020-09-1808:53Saikyunis there a way to get clj working on a machine without access to internet? specifically I'd want clj + project dependencies on a machine behind a very restrictive firewall. I'm able to transfer files to it, and when building I use uperdeps.#2020-09-1808:53SaikyunI know that with leiningen I could download a jar-file and a bat-file, then it didn't need access to internet. is it possible to do something like that with clj?#2020-09-1809:06dominicm@saikyun the installer just downloads a couple jar files and puts them on the filesystem, then puts a script in /bin/ to run that jar.#2020-09-1809:07dominicmThe void linux install script is the simplest version I know of, but others might have good examples too 🙂 https://github.com/SevereOverfl0w/void-packages/blob/master/srcpkgs/clojure/template#L15-L23#2020-09-1809:45Saikyunthanks for the example 🙂#2020-09-1809:07dominicmOne idea might be to run the installer into a directory with the -p flag.#2020-09-1809:07Saikyunthe target computer is on windows btw :x#2020-09-1809:18borkdude@saikyun Alternative if you end up running into issues: with deps.clj it's just a matter of downloading the executable and placing the tools jar in ~/.deps.clj (as documented here: https://github.com/borkdude/deps.clj/issues/20)#2020-09-1809:24Saikyun@borkdude thanks, something like that is what I wanted 🙂 what is the analogue to ~ on windows?#2020-09-1809:24borkdude%USERPROFILE%#2020-09-1809:24Saikyunthanks#2020-09-1809:33borkdude@saikyun Here is one user working around firewalls copying his .m2 deps with bb:
https://twitter.com/MrGung1/status/1305906562746253315#2020-09-1809:34Saikyuncool, thanks#2020-09-1809:35Saikyunwhere do you get deps.exe from?#2020-09-1809:35borkdudeGithub releases: https://github.com/borkdude/deps.clj/releases#2020-09-1809:36borkdudeor install via scoop etc, but since you're having firewall issues, Github releases is probably the easiest#2020-09-1809:37Saikyunyeah, thanks a lot.#2020-09-1809:37Saikyunand it's enough for deps.exe to be on PATH, right?#2020-09-1809:38borkdudeyes#2020-09-1809:38borkdudealso works from cmd.exe, no PS needed#2020-09-1809:38Saikyunokay, cool 🙂#2020-09-1809:38SaikyunI'm getting this error now:
Error building classpath. Failed to read artifact descriptor for org.clojure:clojure:jar:1.10.1
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:clojure:ja
r:1.10.1
#2020-09-1809:39Saikyunthis is what prompted me to try to install tools.deps in an alternative way in the first place#2020-09-1809:39SaikyunI wonder if there's some old stuff lying around, trying to find tools.deps in the directory specified when I ran the .ps1 installation script...#2020-09-1809:39borkdudeprobably there's something wrong with your .m2 stuffs?#2020-09-1809:40Saikyunoh, right, I guess I need to put clojure in the .m2 folder? (e.g. by using -p...?)#2020-09-1809:41borkdudetools.deps is going to the internets when the dep is not yet in .m2#2020-09-1809:41Saikyunokay, that makes sense#2020-09-1810:14Saikyunwhen running clj -P , where do the dependencies end up? I'm looking in ~/.m2 but can't find clojure :S#2020-09-1810:14SaikyunI have clojure as an explicit dependency in deps.edn as well#2020-09-1810:15Saikyunnvm, it was in .m2/org/clojure.. , not .m2/clojure...#2020-09-1812:01Saikyungot it all working by removing my local, existing .m2, then ran clj -P, then I copied my new local .m2 to the firewalled machine and now it works 🙂#2020-09-1812:01Saikyunthanks a lot for the help#2020-09-1812:21borkdudeI think the .m2/repository is also configurable in deps.edn (can't remember it from the top of my head)#2020-09-1812:40alexmillerYes, with :mvn/local-repo#2020-09-1812:47borkdudewhich probably also works with -Sdeps? then that could be used for populating an ad hoc repo @saikyun#2020-09-1812:49alexmillerYep#2020-09-1814:10alexmillerI use this for testing "clean cache" cases all the time#2020-09-1814:10alexmiller(without having to muck with my actual ~/.m2/repository)#2020-09-2206:57Saikyunahh, thanks for the tip! great to know 🙂#2020-09-2206:57SaikyunI’m setting up a dev environment on a pretty locked down production machine, so these things are invaluable#2020-09-2207:00borkdudeOne issue I had while developing on a train is that I forgot to download deps for certain aliases #2020-09-2207:01Saikyunah, good point. will keep that in mind#2020-09-1819:09just.sultanovHi all! Does clojure take MAVEN_OPTS from environment variables? I can’t use keystore with clojure, but it’s work with mvn (directly)#2020-09-1819:14seancorfield@just.sultanov I just grepped the source of brew-install, homebrew-tools, and tools.deps.alpha and didn't see that var in any files so I would say "no".#2020-09-1819:15seancorfieldIt does handle ~/.m2/settings.xml for Maven stuff tho' so maybe you could put credentials in there?#2020-09-1819:24just.sultanovI need to use keystore.jks with self-signed corp certificates. I don’t have problems with credentials. Maven by default pick up settings from .mavenrc and MAVEN_OPTS, but I didn’t see any usages in sources of tools.deps#2020-09-1819:44alexmillerdoesn't use it right now#2020-09-1819:45alexmilleryou can log the request at https://ask.clojure.org#2020-09-2114:43timoWhat is the command to only resolve dependencies with tools-deps and not start a repl? I need it for CI.#2020-09-2114:44alexmillerclj -e nil is sufficient. current prereleases have a new -P (prepare) switch for this purpose.#2020-09-2114:46timothanks!#2020-09-2115:03pmonksFor better logging in multi-step builds, I sometimes do clojure -Srepro -e '(println "Dependencies downloaded")' .#2020-09-2114:47ghadi@timok you'll want clojure -e nil instead of clj if you don't have an interactive terminal#2020-09-2114:47ghadiin CI#2020-09-2114:47timook, cool#2020-09-2114:48timoI never understood the differences though#2020-09-2114:48dominicmClj calls rlwrap. So adds things like up/down.#2020-09-2114:49alexmillerright, sorry#2020-09-2114:50timoawesome, thanks for the answers 👍#2020-09-2202:26alexmillerclj 1.10.1.693 (prerelease) is now available:
• Classpath ordering changes. Classpath will always now be ordered extra-paths, paths, then libs. Libs are ordered by tree depth, then alpha. These changes both revert some ordering changes in the last stable release and go further in defining a reproducible classpath order.
• clj -Stree re-enabled for now while we decide on best way to convey classpath-changing options to -Stree and -Spom
• Windows scripts updated to latest
• This prerelease version should be considered a release candidate for the next stable version! If you try it and have feedback, please leave it here#2020-09-2203:20seancorfield@alexmiller I assume the -Stree restoration is because of this difference?
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "c8bbada9a6ab888c8d9b839c8798e5999dfa9d82f986"}, :content ("[email protected]")}#2020-09-2204:02alexmilleryes, but that's just a symptom of the greater problem, which is how to convey classpath changing arguments to tools like pom and tree without doing it through clj's args#2020-09-2204:03alexmillerDatomic's ion-dev has the same need (for lib bundling)#2020-09-2204:03alexmillerso we're looking at something common for all these but don't want to gate on figuring that out#2020-09-2204:10alexmillerThe -X:deps tree stuff is going to have more features over time for other variants of viewing the tree etc#2020-09-2205:19seancorfieldInteresting. Thanks, Alex.#2020-09-2210:59dominicmI thought that the corfield comma was documented on https://clojure.org/guides/faq but it seems not. Is it documented on the site somewhere? It would be useful for the http://nrepl.org docs to link to as it is tripping users up.#2020-09-2211:05sogaiunot as corfield comma, but there are these: https://clojure.org/guides/weird_characters#_whitespace_character
https://clojure.org/reference/reader#_maps (2nd bullet point)#2020-09-2211:07sogaiuin #parinfer we were talking about use of the comma recently for a variety of things: https://clojurians-log.clojureverse.org/parinfer/2020-09-14/1600045197.010800#2020-09-2219:17ghadihow do people handle bumping the version inside pom.xml with minimal effort?#2020-09-2219:17ghadii'm not beneath sed but I was hoping for something intelligent#2020-09-2219:22alexmillerI've done manual, various Maven plugins, sed#2020-09-2219:22alexmillerthe contrib projects actually have this done automatically via Jenkins Maven integration#2020-09-2219:29seancorfieldI edit it manually as the last step of preparing for a release -- and I edit both the <version> and the <tag> entries to be consistent (the latter just has v at the beginning). I tend to do a global find for a regex that captures the current release, the new (planned) release, and my generic placeholder (e.g., 1.1.next) which I may tag new functionality with in the readme, docs, code, and change log. Then I just work my way through those find results one at a time, and I can eyeball that all matches correctly refer to the new release and the old release as needed.#2020-09-2219:30seancorfield(so I might search for 1\.\1.(next|588|623) for example)#2020-09-2219:55vlaaadUgh, why not just create pom.xml from some tdeps-generated inputs?#2020-09-2219:57vlaaadBumping the version will be a different arg to a pom-generating fn#2020-09-2220:00borkdude@ghadi clojure.data.xml is included in bb, I've done XML rewriting with that#2020-09-2220:04borkdude(here's a script for getting the version out of a pom.xml: https://github.com/borkdude/babashka/blob/master/examples/pom_version.clj)#2020-09-2220:06hugodmakejack updates the pom based on a :version tag in a project.edn file. I haven’t got a task to bump the version yet.#2020-09-2220:07ghadised it is#2020-09-2220:08alexmillerhttps://github.com/cognitect-labs/transducers-java/blob/master/build/release#L26#2020-09-2220:09alexmillerin case it saves you any time... (made for Mac sed)#2020-09-2309:28timoone can use xmllint as well for getting the version out like here: https://github.com/replikativ/hasch/blob/87410246864f2e160cccac3a51f7a7e3139c3c45/.circleci/config.yml#L123 but setting it probably mvn plugin like @alexmiller does in the release script.#2020-09-2314:07lreadhttps://github.com/workframers/garamond offers a way to set the version in pom.xml. If you are using your own versioning scheme you can use its --force-version option.#2020-09-2314:09lreadIn the past I have also used maven to do this type of work, for example to update the tag and version:
mvn versions:set-scm-tag -DnewTag=$(git rev-parse HEAD) -DgenerateBackupPoms=false
mvn versions:set -DnewVersion=$(bin/generate_version) -DgenerateBackupPoms=false
Somehow it bothers me how long it takes maven to do this tho…#2020-09-2315:39alexmilleranyone tried the clj prerelease yet?#2020-09-2316:05vlaaadI haven’t, but I’d like to do it today#2020-09-2316:25alexmiller👍#2020-09-2316:33seancorfieldInstalled it Monday night and integrated it into our dev/test/build pipeline -- everything is working as expected. REPLs, aliases, running tests, building uberjars...#2020-09-2316:53alexmillerThx, good to hear#2020-09-2318:44vlaaadI'm just trying to understand the basis stuff, is this intended that :parents of a thing contains a thing itself?
cljfx/cljfx$jdk11
{:mvn/version "1.7.9"
:deps/manifest :mvn
:dependents [cljfx/cljfx$jdk11 cljfx/cljfx]
:parents #{[cljfx/cljfx cljfx/cljfx$jdk11]
[cljfx/cljfx]}
:paths ["C:\\Users\\Vlaaad\\.m2\\repository\\cljfx\\cljfx\\1.7.9\\cljfx-1.7.9-jdk11.jar"]}#2020-09-2318:44vlaaad(that's an entry in :libs)#2020-09-2318:56alexmilleryou should consider :parents to be undefined for now#2020-09-2318:57alexmillerin other words, that's internal implementation stuff leaking out and I make no guarantee it will be there or be anything in particular :)#2020-09-2318:58vlaaadah, okay. what's not internal there?#2020-09-2318:58alexmillereverything else#2020-09-2318:59alexmillerto your original question, I'm not sure why'd you see that though, if you could send me your deps.edn, I'd like to look at it#2020-09-2319:22vlaaadthis is deps.edn: https://github.com/vlaaad/reveal/blob/master/deps.edn#2020-09-2319:23vlaaadI can reproduce it by using clj without any other args and then loading the basis as described here https://clojure.org/reference/deps_and_cli_prerelease#_basis_injection#2020-09-2319:24vlaaad(clj on windows, if that matters)#2020-09-2319:46alexmillerdoesn't matter. I look at it's a quirk of how classifier libs work in maven. when you have a lib foo and a classifier jar foo$a they technically both use the same pom. in this case, the root jar depends on the classifier jar, which has the same pom, so it looks like it depends on itself.#2020-09-2319:47alexmillerso, expected but weird :)#2020-09-2319:53vlaaadah, I see, thanks for clarification!#2020-09-2315:43delaguardohttps://github.com/DeLaGuardo/setup-clojure
btw, github action support prerelease as well. Might make the way to try it a little bit easier 😄#2020-09-2316:52ghadiworks beautifully @delaguardo. Thanks so much!#2020-09-2317:01seancorfield@delaguardo I'm using your setup for GH Actions on several of my projects so "Thank you!" -- one comment: using tools-deps for the CLI version is a bit confusing since tools.deps is a library with a different version numbering schema -- Clojure CLI 1.10.1.693 uses tools.deps 0.9.799 for example.#2020-09-2408:32delaguardothanks for the feedback!
I made a PR to address that confusion:
https://github.com/DeLaGuardo/setup-clojure/pull/13/files#diff-cdc05b01d72867060bc4d72cb108cf25
how do you think about it?#2020-09-2415:38seancorfieldThat looks good!#2020-09-2415:53delaguardomerged, thanks again for pointing on that#2020-09-2320:21grzmWhere might addressing https://clojure.atlassian.net/browse/TDEPS-46 “Repositories from dependencies’ pom.xml are not used” be on the horizon? Any pointers on where one might get started on fixing it?#2020-09-2321:53alexmillerHey, sorry I didn't catch your question earlier in the flow...#2020-09-2321:54alexmillerI am honestly reluctant to fix this the more I think about it and talk to others about it#2020-09-2321:55alexmillerThere are a lot of security concerns around getting someone's else repository, which may either be an internal repo or a repo that provides rogue versions of deps defined in central repos#2020-09-2321:55alexmillerSo my recommendation right now would be to add those repositories explicitly to your deps/pom instead#2020-09-2322:02grzmThanks for the feedback! We ran into this today, so I wanted to see what the current thinking on this is. Cheers!#2020-09-2320:25seancorfield@alexmiller We haven't seen any impact of the changes to classpath order (yay!) but that made me wonder about pom.xml generation: is the order of the <dependencies> section there important, from a Maven p.o.v.? i.e., does it affect the order things show up on the classpath and does clojure -Spom generate the <dependencies> section in a reproducible order now based on the classpath?#2020-09-2321:01alexmillergood question. the order is important to maven (you should get deps in the order listed in the pom) and I have not explicitly looked at that ordering wrt deps#2020-09-2321:07vlaaadso that might be unstable if deployed library is used in a project that uses maven as a build tool?#2020-09-2321:11alexmillerwell, deps.edn takes its deps in a map so from a clj pov, there is no order set by the user (the new changes order by alpha)#2020-09-2321:12alexmiller@seancorfield if you wanted to file a jira about this, I'd be happy to look at it#2020-09-2322:14seancorfield@alexmiller I don't care enough about the issue to file a Jira ticket. I was mostly just curious if you knew off the top of your head whether -Spom generated <dependencies> in the same order as they would now appear on the classpath.#2020-09-2322:14seancorfield(I didn't care much about the non-reproducible classpath ordering before either -- although some others did)#2020-09-2322:17alexmillerI looked at the code and the answer is no - they are not necessarily in the same order they'd appear on the classpath#2020-09-2405:38andy.fingerhutThis may be a distraction from real tools.deps work, but was wondering if others had come across a series of articles by Russ Cox on Golang and package versioning. This is one of those articles I have looked at recently: https://research.swtch.com/vgo-principles. One thing I found interesting is the number of different package managers that use SAT solvers to find what collection of modules and their versions to use. In case you are curious, Russ argues that maybe this is a bad idea.#2020-09-2413:05alexmillerYeah I’ve read these. Some of the Go stuff is similar to the tools.deps resolution#2020-09-2415:00andy.fingerhutI've only read a couple of Russ Cox's articles, but he seems to do a thorough job of writing about pros and cons of various approaches, with examples.#2020-09-2504:46andy.fingerhutOut of curiosity, did you consider the go module version approach of using the minimum version among all requests for a dependency, rather than the latest, in Clojure CLI tools? It sounded unintuitive at first, but the examples in those articles do seem to make it pretty clear that it is at least one way to achieve better reproducibility in versions of libraries used by an application.#2020-09-2504:47alexmillerbleh#2020-09-2504:49alexmillerif you are developing for growth, not breakage, that doesn't make sense#2020-09-2514:49andy.fingerhutGo has apparently strongly recommended backwards compatibility in APIs and behavior since 2013, before Spec-ulation talk, for similar reasons. I think the "largest of the minimum required versions" of this latest Golang vgo tool is fairly reasonably argued for on the basis of reproducible builds, regardless of whether someone releases a new version of a lib tomorrow. Anyway, not trying to change Clojure CLI tools in this regard.#2020-09-2406:47vlaaadI like the new exec stuff.
User deps.edn :
{:aliases {:reveal {:extra-deps {vlaaad/reveal {:mvn/version "1.0.130"}}
:ns-default vlaaad.reveal}}}
Starting a repl is as simple as clj -X:reveal repl
❤️#2020-09-2406:55sogaiuyes, i liked that too.
was hoping i could stack -X to use multiple tools (like one could with -A), but i get the impression that's not part of the design intent.#2020-09-2406:56vlaaadI wonder if that can be done using injected basis#2020-09-2407:03sogaiuhaven't thought about that angle yet, may be there's something to that.
i did think one might be able to make one thing that's designed to do the stacking for anything else that might follow its conventions.#2020-09-2410:37practicalli@vlaaad I would encourage people creating tools for Clojure CLI to support theuse the -X flag. As well as using key value pairs for clarity you can also set default values, making the comand line even simpler
As example, using Sean's clj-new project and the alias below I can create a library project with a default name by just running the command clojure -X:project/new
:project/new
{:extra-deps {seancorfield/clj-new {:mvn/version "1.1.215"}}
:exec-fn clj-new/create
:exec-args {:template lib :name practicalli/playground}
:main-opts ["-m" "clj-new.create"]}
I can then choose to over-ride value in the :exec-args default values by including key value pairs on the command line
clojure -X:project/new :name practicalli/banking-on-clojure#2020-09-2410:42vlaaadI mean that's exactly what I do :)#2020-09-2410:46practicalliSo I could have an alias similar to this, that would let me do clojure -X:inspect/reveal to run a repl and clojure -X:inspect/reveal :command prepl to run prepl instead
:inspect/reveal
{:extra-deps {vlaaad/reveal {:mvn/version "1.0.130"}}
:exec-fn vlaaad.reveal
:exec-args {:command repl}}
Does that make sense?
To support versions before the pre-release you could also include main-opts
:inspect/reveal
{:extra-deps {vlaaad/reveal {:mvn/version "1.0.130"}}
:exec-fn vlaaad.reveal
:exec-args {:command repl}
:main-opts ["-m" "vlaaad.reveal" "repl"]}
#2020-09-2411:10vlaaadthere is no vlaaad.reveal fn#2020-09-2411:10vlaaadyou can use :exec-fn vlaaad.reveal/repl#2020-09-2411:13vlaaadmy example earlier used ns-default because there are various types of repls that I might want to start, so I want that to be a command line option — it doesn’t prevent :exec-fn use#2020-09-2411:18vlaaadI don’t see why there should be a single top-level function that receives :command as argument, exec-fn already solves the problem of calling different functions in the ns. Various repls in vlaaad.reveal ns are independent functions with their own different arguments, smashing them together would be unnecessary complecting#2020-09-2411:42practicalliI was only suggesting an example of a general form of alias, along the lines of the clj-new as that is all I am familar with at the moment. Looking forward to see what develops with reveal.#2020-09-2412:04sogaiunice pun 😉#2020-09-2412:43vlaaadyeah, I’ll think of a more versatile suggested alias to put into reveal docs once -X is no longer pre-release :+1:#2020-09-2410:46dominicmA lot of tools support later command flags overriding earlier ones anyway 🙂#2020-09-2412:13sogaiui have something for starting a socket repl on a specific port.
invocation is like:
clj -X:alc.socket-repl :port 9123
there is also a mode where you don't specify a port and it tries to choose one for you:
clj -X:alc.socket-repl
Socket repl port: 38735
Clojure 1.10.1
user=>
here it will print the chosen port number out so it's less digging around you have to do to get this info to your editor or whatever other tooling.
it also makes the port info available programmatically -- jcmd <pid> VM.system_properties should have it show up in the way clojure arranges for it w/ clojure.server.repl=... (i have a separate tool that can obtain this info)
at the end it just hands things off to clojure.main/main with --repl.
i wish i could use this along with other folks' things -- so far -X doesn't look to be composable. what i should say is not necessarily that, but some way to compose things -- it doesn't have to be via -X.#2020-09-2415:44seancorfieldComposability of "main" functions is problematic though: there's no guarantee a "main" will exit (if it starts a server, for example) and aliases in general are merged to produce the final configuration that is used to start a (single) program.#2020-09-2415:46seancorfieldWhat has been suggested (several times, here in this channel) is to write your own "runner" function that takes a list of functions to execute and runs them one after the other. Now that programs can get at the basis easily (if they are run via the CLI tools -- not if they are run via a JAR!), you could pass a list of aliases and the "runner" could walk that list and exec each alias...#2020-09-2415:47seancorfield...you wouldn't get the same semantics of composition but you would get "multiple main" functions run (and you could control the semantics yourself).#2020-09-2415:48seancorfield(this is exactly why we have a build script that wraps the CLI: you can specific multiple "commands" to build and it runs the CLI for each command, by way of a shorthand)#2020-09-2422:33alexmillerhas anyone tried the clj prerelease 1.10.1.693 on windows?#2020-09-2422:34alexmillerI've done a fair amount of testing with it, but would be great if someone else had used it#2020-09-2423:05sogaiuin case others want to try and don't want to look for instructions (https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows):
0) remove previous installation (for me this was deleteing the ClojureTools folder under Documents\WindowsPowerShell\Modules)
1) download: https://download.clojure.org/install/win-install-1.10.1.693.ps1
2) did the "unblock" thing via properties (right-click on file's icon in windows explorer and choose properties)
3) execute the ps script via powershell
4) choose 2 (for under Documents\WindowsPowerShell\Modules) and wait a bit
5) clj -h to verify version
6) freshly clone a clojure project that uses deps.edn
7) start it via clj
something close to those steps worked for me.#2020-09-2519:00seancorfield@alexmiller Now that 1.10.1.697 is a stable release, any more thoughts on this discrepancy?
(! 740)-> clojure -Sforce -Stree
org.clojure/clojure 1.10.1
org.clojure/spec.alpha 0.2.176
org.clojure/core.specs.alpha 0.2.44
(! 741)-> clojure -Sforce -X:deps tree
org.clojure/clojure 1.10.1
org.clojure/core.specs.alpha 0.2.44
org.clojure/spec.alpha 0.2.176
org.clojure/tools.deps.alpha 0.9.810
javax.inject/javax.inject 1
com.cognitect.aws/api 0.8.408
... lots more stuff ...#2020-09-2519:03seancorfield(I verified we do not see the same discrepancy with -Spom vs -X:deps mvn-pom)#2020-09-2519:05alexmillerI'll get to it#2020-09-2519:05alexmillerhave not had a chance to finalize the plan for it yet#2020-09-2519:06alexmillerfor now, -Stree exists#2020-09-2519:29alexmillerClojure homebrew tap has the latest of course. Homebrew core and linux core should get this automatically now, but I have no idea how long that will take, looking forward to watching that :)#2020-09-2614:25sogaiu@borkdude it's a minor thing but doing deps -h gives the version string twice here:
$ deps -h
Version: 1.10.1.697
Version: 1.10.1.697
...
#2020-09-2614:27borkdudeok, going to fix now#2020-09-2614:44borkdude@sogaiu Fixed (and all remaining open issues as well). If there's nothing else, going to release#2020-09-2614:56borkdudeReleased: https://github.com/borkdude/deps.clj/releases/tag/v0.0.10#2020-09-2617:50practicalliI've updated the practicalli/clojure-deps-edn GitHub project and replaced all uses of -A and -R flags for using all aliases, substituting them for -M without any issues... it actually makes things simpler 🙂
Details are in #announcements
Thank you Alex (and Sean)#2020-09-2619:24dominicmExecution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/run/exec__init.class, clojure/run/exec.clj or clojure/run/exec.cljc on classpath.
Full report at:
/tmp/clojure-16205359177864566580.edn
Presumably this means I need to correct something in the void installer, what is that?#2020-09-2619:25borkdude@dominicm Is this the normal clojure CLI?#2020-09-2619:26dominicmAh, a new exec.jar :)#2020-09-2619:27dominicmOK, made that part of the new void installer :)#2020-09-2619:27borkdudewhat's a void installer?#2020-09-2619:43dominicmVoid Linux installer :)#2020-09-2619:27borkdude@alexmiller Feedback on error message with the new exec:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "ddbfb2afb6b9a8b9b89d909f8defedece4"}, :content ("[email protected]")}
I think the latter could be improved#2020-09-2619:36alexmillernot sure I understand why that's failing#2020-09-2619:36borkdudebecause there's no val to the key#2020-09-2619:36borkdudeI think#2020-09-2619:37alexmilleroh, that's probably it#2020-09-2619:37alexmillerit looks at whether the non-alias arg count is even or odd#2020-09-2619:37alexmillerthere is some fiddly code there trying its best to guess what's wrong#2020-09-2619:38borkdudecan aliases be whitespace delimited?#2020-09-2619:38alexmillerno#2020-09-2619:38borkdudethen where is the ambiguity?#2020-09-2619:38alexmillerI don't think there is here, just saying that's the code at faultl#2020-09-2619:39borkdudeah ok. want me to make a JIRA?#2020-09-2619:39alexmillersure#2020-09-2619:43borkdudehttps://clojure.atlassian.net/browse/TDEPS-168#2020-09-2621:29vlaaadbtw, real world example where this might be super confusing: I tried passing var as an argument (`#'inc`), but shells treat # as a start of a comment, so...#2020-09-2623:59alexmillerYeah, I’ll fix#2020-09-2619:40alexmillerthere was a period of time where other variants were accepted, this might be a legacy of that, don't remember#2020-09-2619:40borkdudeUnrelated. If checking in binaries into git wasn't an anti-pattern, I think Clojure git deps could be used to implement an ecosystem of binaries more or less like brew, but using deps.edn. The classpath can then be used as the binary path, a bit similar to a nix env. I would have a purpose for this (babashka pods). The git deps stuff could be made to work with GraalVM as well I think, so everything could be very fast after deps have been downloaded (but even without GraalVM, it's already fast due to classpath caching).#2020-09-2622:19sogaiui experimented with this once using git lfs.#2020-09-2622:20sogaiuthe binaries in git thing and finding things under ~/.gitlibs that is#2020-09-2622:22borkdudeah interesting. what did come out of it?#2020-09-2622:26sogaiuwell for one thing i think i learned that not everyone has git lfs 🙂#2020-09-2622:26sogaiuthough it looks like typical git installs on windows get it#2020-09-2622:27sogaiuother than that it seemed to work pretty well#2020-09-2622:28sogaiuin addition to the size aspect of the repository, trying to but binaries for multiple platforms in one repository didn't feel great because the end user then gets extra cruft.#2020-09-2622:28borkdudeHere I'm sketching out an idea: https://github.com/babashka/babashka.pods/issues/19
One option is to have a .edn file inside the git dep which refers to the download#2020-09-2622:29sogaiuyeah, i saw that page first 🙂
the .edn file inside idea seems preferable to the binaries in the repository idea#2020-09-2622:29borkdudeyeah. also the binaries could be pulled the first time you are trying to find them#2020-09-2622:30borkdudethis idea could even work for non-gitlibs as well#2020-09-2622:30borkdudemaybe there could be a nice coupling with github releases somehow#2020-09-2622:30sogaiuwhat would the ultimate location of the binaries be?#2020-09-2622:31borkdudethe resolve function will download the link and the put them inside .foobar/your/dependency/<sha>/the-binary and then that binary file will be invoked#2020-09-2622:32borkdudewhere .foobar is some name for this package manager layered on top of tda#2020-09-2622:36sogaiuah ok, thanks for the explanation -- i see it at the issue 🙂#2020-09-2619:40borkdudeWonder what others think. Maybe instead of checking in the binaries themselves, there could be links to releases in the source, with checksums, etc. But most of the gnarly dependency stuff could be handled by tda.#2020-09-2619:49borkdudeImplementing something like brew this way would only require a couple of tens of lines of code maybe.#2020-09-2619:58borkdudeThis could be a package manager for macOS, linux and Windows at the same time#2020-09-2620:32borkdudeAha: https://github.com/aerys/gpm - unclear how it resolves deps#2020-09-2621:33borkdudeMade an issue here for further discussion: https://github.com/babashka/babashka.pods/issues/19#2020-09-2711:54borkdudeImplementation of the above idea: https://github.com/borkdude/cpm#2020-09-2712:10sogaiu@borkdude for linux, i changed the .cpm.edn file for babashka to be:
{:package/name org.babashka/babashka
:package/description ""
:package/version "0.2.1"
:package/license ""
:package/artifacts
[{:os/name "Mac.*"
:os/arch "x86_64"
:artifact/url ""
:artifact/executables ["bb"]}
{:os/name "Linux.*"
:os/arch "amd64"
:artifact/url ""
:artifact/executables ["bb"]}]}
then i was able to follow your readme.#2020-09-2712:11borkdudePR welcome :)#2020-09-2712:11sogaiulol, ok, i'll send one 🙂#2020-09-2712:14borkdudeI pushed an update to bb, now cpm is also executable using bb:
$ bb -cp src:test-resources -m cpm.main clj-kondo
/Users/borkdude/.cpm/packages/clj-kondo/clj-kondo/2020.09.09
which is fast :)#2020-09-2712:21sogaiuindeed:
$ time bb -cp src:test-resources -m cpm.main clj-kondo
/home/user/.cpm/packages/clj-kondo/clj-kondo/2020.09.09
real 0m0.015s
user 0m0.009s
sys 0m0.006s
#2020-09-2718:54dominicmVoid Linux has now updated to clj 697 😃#2020-09-2808:49viestihumdum, I sometimes get this error in CirleCI while building project:
clojure -A:... -m kaocha.runner analysis
Picked up JAVA_TOOL_OPTIONS: -Dclojure.main.report=stderr -XX:-OmitStackTraceInFastThrow
Cloning:
using cimg/clojure:1.10.1 as image, with Clojure cli version 1.10.1.536#2020-09-2808:50viestimight be totally relared to this project only, but just thinking out loud, that might there be a concurrency issue in downloading dependencies#2020-09-2813:09alexmillerYou could try adding -Sthreads 1 and see if that helps#2020-10-0208:09viestithanks! 🙂#2020-09-2815:58jjttjjI just updated my clj tools to the latest version and am now getting
CompilerException java.lang.RuntimeException: Unable to resolve symbol: requiring-resolve in this context, compiling:(cljfx/context.clj:37:32)
which would seem to imply that an older version of clojure being used? Is there an obvious reason this would happen after the clj update? (Both cljfx and my project seem to explicitly depend on clojure 1.10.1)#2020-09-2817:34jjttjjOk I figured this out. I had a :local/root dependency which points to a standalone jar file which uses clojure 1.9, which seems like it causes the repl started from clj to be 1.9. Is this expected?#2020-09-2816:04borkdude@jjttjj clojure -Stree is your friend here#2020-09-2816:05borkdude(or whatever is the way of doing the tree now)#2020-09-2816:49alexmillerStill that for now#2020-09-2816:10jjttjjHmmm it looks like clj is giving me a clojure 1.9 repl even though my project only has 1.10 in the tree. might be a scoop issue going to try reinstalling clj
Edit: ok I think this is a scoop issue#2020-09-2816:37natea couple of things I've noticed that are different about the new cli tools:
- Only the first -M is taken. I used to be able to do -A:one -A:two before and both :one and :two were used (was useful for a script I had that wrapped a call to clojure and I could add extra args if I wanted)
- If I put -Spath at the end of my command, it's ignored if there's a -M.... Specifically clojure -Spath -M:repl prints a classpath, but clojure -M:repl -Spath runs the main specified in the :repl alias#2020-09-2816:50borkdude@nate -M:one:two#2020-09-2816:50alexmiller-A and -M have always supported multiple aliases so -M:one:two is what you need there. -M and -X should now be last as they grab all args afterwards#2020-09-2816:51alexmillerSo clojure -Spath -M:repl#2020-09-2816:52nateright, I know that you can put them together, my point is that you used to be able to supply them separately#2020-09-2816:52alexmillerNow you can’t #2020-09-2816:52nateI definitely understand why the behavior changed#2020-09-2816:52natecool#2020-09-2816:53alexmillerMultiple -A should still work#2020-09-2816:53nateinteresting point about -A#2020-09-2816:54natedeprecation warning is pushing me away from that, for good reasons#2020-09-2816:54borkdudeclojure -A:foo -M:bar should also work I think? brings in deps/paths from foo and bar but executes main from bar?#2020-09-2816:54alexmilleryes#2020-09-2816:55seancorfieldThere are a lot more changes in this version than I think are perhaps obvious on the surface -- it's probably always been "best practice" to specify :main-opts separately via a single -M option after using one or more -A options to accumulate aliases but that's never really been codified/written down. This new release sort of "enforces" (or at least "strongly encourages") what I think the core team have always considered best practice...#2020-09-2816:56nateyeah, I agree with that#2020-09-2816:56seancorfield-M used to only bring in :main-opts but now brings in paths/deps as well which should not cause any surprises -- but it is quite an important difference.#2020-09-2816:57borkdudeare you referring to a previous pre-release or release?#2020-09-2816:58seancorfieldI'm talking about this in the context of "previous stable" and "current stable" releases.#2020-09-2816:58nateme too#2020-09-2816:58borkdudeI didn't even know there was an -M in a previous release#2020-09-2816:59borkdudeA minor thing but -M -m still feels weird to me :)#2020-09-2817:00alexmillerconsider it encouragement to use -X :)#2020-09-2817:01seancorfield"old" CLI reference https://github.com/clojure/clojure-site/blob/58b9c790052c0d2dce8aeb1e3cd0872bd9dac296/content/reference/deps_and_cli.adoc#aliases shows -R, -C, -O, -M, and -A @borkdude#2020-09-2817:03borkdude-X is ok but I don't consider it a substitute for a command line interface for general usage#2020-09-2817:04borkdudei.e. there will still be a place for tools.cli based tools#2020-09-2817:04borkdudeunless I'm missing something#2020-09-2817:11borkdudeBut I guess the -m gets hidden in an alias mostly anyway :)#2020-09-2817:12tvaughanI was doing something like clojure -A:alias-with-main-opts -m some.other.main With the new release I get a deprecation warning to switch to -M, but with -M -m some.other.main are passed as command-line options to the main entry point associated with alias-with-main-opts. Does this mean there's now no way to override a main entry point?#2020-09-2817:13seancorfield@tvaughan I believe clojure -A:alias-with-main-opts -M -m some.other.main would be the equivalent now.#2020-09-2817:14seancorfield(I haven't tried that...)#2020-09-2817:18tvaughanThanks @seancorfield. Interestingly, this clojure -A:toolbox:dxf-tools -M -m server.core results in an error message from my application about -m not -M being an unknown option. The main-opts associated with dxf-tools are still being used#2020-09-2817:19seancorfieldHmm, sorry. Guess I'll have to do some digging (unless @alexmiller can give a quick answer off the top of his head).#2020-09-2817:20tvaughanNo worries. I appreciate the help. I can dig further too. I took a quick look at the changelog but didn't see anything about this specifically#2020-09-2817:23seancorfieldAh, I think this is one of those cases where the advice is to split up :alias-with-main-opts into two aliases -- one with :main-opts (and any deps needed solely for that) and one with all the (other) deps.#2020-09-2817:24seancorfield(the other option is to continue using the -R and/or -C options even though they are deprecated)#2020-09-2817:26tvaughanCool. Thanks. I thought that might be the case. Didn't want to change things without confirmation tho :)#2020-09-2817:46alexmillerif you're including an uberjar in your classpath, then you've broken the ability of tools.deps to properly resolve libs#2020-09-2817:46alexmillerb/c it can't "see" inside the uberjar to tell what libs are included there or their versions#2020-09-2817:52jjttjjThat makes sense#2020-09-2818:15seancorfield@alexmiller Now that 1.10.1.697 is the stable release, there are four mentions of clj -m <some.ns> in https://clojure.org/guides/deps_and_cli that need to be updated to include -M if you want folks to avoid the clojure.main warnings (i.e., "WARNING: When invoking clojure.main, use -M").#2020-09-2820:02alexmillerYeah it’s on my list to update this week#2020-09-2818:15seancorfieldThose all need to be clj -M -m <some.ns> now, right?#2020-09-2818:23seancorfield(also, I added a note on TDEPS-100 that indicates some confusion over -A since the reference page does not define clj-opt and clojure -h says that -A is an exec-opt which seems to be at odds with the CLI/deps reference text)#2020-09-2819:21borkdudeyes, now I feel the issue we discussed last weekend. I have this:
{;;:paths ["your-packages"]
;;:deps {your-org/your-packages {,,,}}
:aliases
{:glam {:extra-deps
{borkdude/glam {:git/url ""
:sha "4599fb019deae9418d76a9996ae19b4003f3cc96"}}
:main-opts ["-m" "glam.main"]}}}
and now I want to do:
bb -cp $(clojure -R:glam -Spath) -m glam.main --install clj-kondo/clj-kondo --global --verbose
but I have to make two aliases now, so from Clojure this becomes:
clojure -M:glam-deps:glam
:/#2020-09-2819:22borkdudewith -Spath it's pretty clear you don't want to run the main right#2020-09-2819:22borkdudeso why not ignore the main when doing this:
clojure -A:glam -Spath
#2020-09-2819:23borkdudethat doesn't seem to be breaking#2020-09-2819:33borkdudeah well, bash to the rescue:
{:aliases
{:glam/packages {:extra-deps
{borkdude/glam {:git/url ""
:sha "4599fb019deae9418d76a9996ae19b4003f3cc96"}
;; your-org/your-packages {,,,}
}
;; :extra-paths ["your-packages"]
}
:glam {:main-opts ["-m" "glam.main"]}}}
alias glam='clojure -M:glam/packages:glam'
$ glam --install clj-kondo/clj-kondo org.babashka/babashka
#2020-09-2820:03seancorfield@borkdude If you put -Spath first it will not run the main.#2020-09-2820:04borkdudeah really#2020-09-2820:04borkdudethanks!#2020-09-2820:05seancorfieldAnd that's supposed to be clear from the docs which show clj [clj-opt*] [exec-opt] as the syntax -- -Spath is a clj-opt, -M is an exec-opt#2020-09-2820:05borkdudeyep, makes sense, thanks for clearing that up#2020-09-2820:05seancorfield(as I just noted to @alexmiller it's not clear whether -A is meant to be a clj-opt or an exec-opt at this point, esp. given that at some future date it will stop running :main-opts)#2020-09-2820:06alexmillerIt’s kind of in a gray area#2020-09-2820:06alexmillerBut I think I would call it a clj-opt#2020-09-2820:10seancorfieldAt this point, I'm hoping that -A stops running :main-opts fairly soon and it can be clearly classified as a clj-opt and then -R and -C can be fully removed.#2020-09-2820:33borkdude@seancorfield I'm still getting the warning:
$ clojure -Sforce -Spath -A:glam
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
#2020-09-2820:33borkdudeI don't want main, I want path#2020-09-2820:33borkdudeI used to use -R -C for this#2020-09-2820:34borkdudecan I rely on this being a correct invocation and the warning begone next version?#2020-09-2820:34seancorfieldclojure -Sforce -Spath -M:glam should work.#2020-09-2820:34seancorfield-Spath implies -P I believe.#2020-09-2820:34borkdudeit works, but I get the warning#2020-09-2820:35seancorfield(! 696)-> clojure -Sforce -Spath -M:runner
src:resources:/Users/sean/.gitlibs/libs/com.cognitect/test-runner/f7ef16dc3b8332b0d77bc0274578ad5270fbfedd/src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar:/Users/sean/.m2/repository/org/clojure/tools.namespace/0.2.11/tools.namespace-0.2.11.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
#2020-09-2820:35seancorfieldWithout -Spath that runs the tests (or attempts to -- it's missing the :test alias)#2020-09-2820:36borkdudeaha, with -M it doesn't warn, but with -A it does... that's weird#2020-09-2820:36borkdudeI think -A is the preferred way for creating paths right#2020-09-2820:36seancorfield(! 698)-> clojure -Sforce -M:test:runner
Running tests in #{"test"}
Testing tvaughan.example-test
FAIL in (a-test) (example_test.clj:7)
FIXME, I fail.
expected: (= 0 1)
actual: (not (= 0 1))
Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
(! 699)-> clojure -Sforce -Spath -M:test:runner
test:src:resources:/Users/sean/.gitlibs/libs/com.cognitect/test-runner/f7ef16dc3b8332b0d77bc0274578ad5270fbfedd/src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/test.check/1.0.0/test.check-1.0.0.jar:/Users/sean/.m2/repository/org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar:/Users/sean/.m2/repository/org/clojure/tools.namespace/0.2.11/tools.namespace-0.2.11.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
(! 700)->
🙂#2020-09-2820:37borkdudeanyway, I'll be using -A (and advocating it in my README) and expect the warning to be gone some time in the future#2020-09-2820:40borkdudeclojure -Sforce -Spath -A:glam 2>/dev/null
#2020-09-2820:42borkdudeFeel free to comment on this README for improved clojure usage in case I missed something: https://github.com/borkdude/glam#2020-09-2915:53borkdudefyi, the above is a cross-platform package manager for binaries written using Clojure, Windows support is coming soon (with the intent of not using Powershell)#2020-09-3021:52markbastianIs there a way to ignore the global ~/.m2/settings.xml when resolving repos? I have a situation in which my global file has a forced mirror that doesn't seem to be resolving snapshots correctly but if I just add the :mvn/repos to my deps.edn file and mv settings.xml settings.bak it works.#2020-09-3021:53markbastianBut I don't want to have to do that every time I want to sync my snapshots to my internal repo.#2020-09-3022:12alexmillerThat’s maybe the best option right now#2020-09-3022:12alexmillerYou could also set that repo id to nil in your deps.edn and then add a new one?#2020-09-3022:23markbastianI’ll give that a shot. I’ve also got another workaround in which I run the clj command in a docker image with the source and ~/.m2/repository mounted in the container. Thanks!#2020-10-0105:53pmonksApologies if this has been asked & answered before (I did search, but couldn’t find anything relevant), but in recent versions (specifically v1.10.1.697) the following occurs:
$ clj -e '(println "Hello, world")'
WARNING: When invoking clojure.main, use -M
Hello, world
$
I understand that this can be “fixed” with clj -M -e '(println "Hello, world")' , but is that intentional?#2020-10-0108:27practicalliIt seems so. The -M flag is required when invoking clojure.main.#2020-10-0116:26pmonksThat seems like an unfortunate UX decision…#2020-10-0113:09alexmilleryes, all invocations of clojure.main should now be routed through -M#2020-10-0113:10borkdudeeven when $ clj -- -e '(+ 1 2 3)' which is quite unambigious?#2020-10-0113:11alexmillerit's unambiguous now but we're trying to open up arg space for repl execution#2020-10-0116:28pmonksIt seems to me that evaluation of a something provided on the command line should be considered at the same level of importance as running a REPL or invoking a main function, FWIW.#2020-10-0117:16alexmillerexpression eval is done by invoking clojure.main. -M is how you (now) invoke clojure.main.#2020-10-0117:21vlaaadHow are we going to invoke clojure.main in the future? :)#2020-10-0117:21alexmiller-M#2020-10-0117:22alexmillerwhere future = now#2020-10-0117:34vlaaadAh, I misunderstood your previous message#2020-10-0117:35alexmilleryeah, I meant vs previous, not vs future#2020-10-0117:34alexmillerclj was originally built solely around clojure.main. we are in the process of making it bigger than that and this is part of segmenting off that part of the functionality.#2020-10-0118:21pmonksI understand the improvements that brings to the implementation, I’m more making a comment about the (negative, imho) UX that creates, especially for newcomers.#2020-10-0118:21pmonksi.e. why should a newcomer have to know that in order to evaluate an expression on the command line, they are actually invoking clojure.main? That seems like incidental complexity to me…#2020-10-0118:22pmonksTo non-Java/Clojure folks, clj is clojure.main.#2020-10-0118:22alexmillerI don't think evaluating an expression on the command line is actually a thing most people do compared to running a repl#2020-10-0118:25pmonksSure, but that doesn’t mean that CLI evaluation isn’t worth supporting in a clear and concise (and backwards compatible, though I know we’re in a period of time where that is being consciously and deliberately broken) manner.#2020-10-0118:30alexmilleris your complaint about the warning message mentioning clojure.main?#2020-10-0118:30pmonksNo.#2020-10-0118:31alexmillerok, then what is the complaint?#2020-10-0118:31pmonksMy complaint is why does anyone need to know they’re invoking clojure.main in order to evaluate something on the command line. Shouldn’t this be a “first class” operation, regardless of how it’s implemented under the hood?#2020-10-0118:32pmonksAfter all that’s how most other languages handle this - Principle of Least Surprise etc. etc.
$ python -c 'print("Hello, world")'
Hello, world
$ perl -e 'print "Hello, world"'
Hello, world
etc.#2020-10-0118:37alexmillerok, thanks for the feedback#2020-10-0118:39pmonksAnd to be clear, I don’t do this all that often, but it is a handy feature (e.g. in shell scripts).#2020-10-0118:40alexmillerthe most common use of it that I'm aware of is to trigger deps loading without doing something else, which you can now do with -P#2020-10-0122:19pmonksSure, but -P won’t evaluate an arbitrary expression. I like to (println a specific message to make build log searching easier.#2020-10-0118:38borkdude> I don't think evaluating an expression on the command line is actually a thing most people do compared to running a repl
I do this all the time on the command line#2020-10-0118:38borkdude-M -m -M foo.clj -M -e, don't know if I can get used it ;)#2020-10-0118:38alexmillerwhy vs running a repl?#2020-10-0118:40borkdudeA lot of things I do in my open source projects is testing out command line applications which are usually one shot. Yes, I also use a REPL, but actually more often I execute one time invocations#2020-10-0118:41alexmillerso is that -e or -m ?#2020-10-0118:41borkdudelet me just do a zsh history thing#2020-10-0118:42alexmiller(note new -P to prepare deps w/o executing, which is one prior use case for -e)#2020-10-0118:52borkdudeok, I wrote this horrible script now:
#!/usr/bin/env bash
history | bb -iO '(->> *input*
(map #(as-> % x (str/trim x) (str/split x #"\s+") (rest x) (take 2 x)))
(keep #(when (and (str/starts-with? (first %) "cl") (second %) (> (count (second %)) 1)) (subs (second %) 0 2)))
frequencies (sort-by second >) (take 10))'
["-A" 326]
["-S" 208]
["-e" 172]
["--" 79]
["-M" 76]
["/t" 30]
["-J" 22]
["-m" 18]
["-X" 12]
["-R" 8]
which indicates I use -e way more often on the command line#2020-10-0118:52borkdudebut so far I haven't used -M a lot, usually -A#2020-10-0118:53borkdudebecause instead of -m there's usually an alias for it. But with -e there isn't. So when invoking main directly, -e is my primary use case.#2020-10-0118:58alexmillerthanks for the data#2020-10-0119:03borkdudeI think clj /tmp/foo.clj is also one of my primary use cases, this is where ["/t" 30] probably comes from#2020-10-0119:03borkdudebut because the filenames can change, it doesn't really show well here#2020-10-0119:05seancorfieldI just checked my history, my dot-clojure file, and my deps.edn at work and I almost never use -e without also using -m it seems. Just as a contrary data point to the above.#2020-10-0119:06alexmillerthx#2020-10-0119:07borkdude$ alias cljm='clojure -M'
$ cljm -e "(+ 1 2 3)"
6
it works#2020-10-0120:29practicalliFor my own brief experiences, I only every used -e once as a saw an example in the Clojure CLI docs. I am not aware of any use cases where I would want to evaluate clojure on the command line, especially if I can run the repl just as quickly. If I was going to script something with Clojure, then I'd try using Babashka, it looks great. I am fascinated as to what the scenarios could be for using the -e . If anyone has a link to examples, please let me know.
The only thing I find a little strange is when running an application from the command line, eg. clojure -M -m namespace.main , from a usability point of view the -M looks a little redundant.. After thinking about this for a few days, I realize I don't actually run the application that way. I'll either run in the REPL or create a jar/uberjar and run it that way. I though about using an alias called :project/run that ran the unpackaged project, which I could include in a project template (eg. for clj-new), but not sure how valuable that would be.
I find the -X flag more appealing (and more Clojurey) approach, as I've been writing command line -main functions to take hash-map as an argument for a while now. It seems an idiomatic approach to have all entry point functions in a project take a hash-map where possible.#2020-10-0120:31dpsuttonin fish i have these handy shell functions
function clj-doc --description 'Get the docstring of a function'
clj -e "(require 'clojure.repl)(clojure.repl/doc $argv[1])"
end
one clj-doc and the other clj-source. easy to update to -M -e though#2020-10-0120:31borkdudeWell, sometimes I use -e to require some namespace or a compile invocation for example#2020-10-0120:32borkdudeNo big deal, can be easily hidden in a script, Makefile or whatever, I just don't find -M -m aesthetically pleasing, which may not be an important argument. Whatever :)#2020-10-0122:20lreadFor what it is worth, I am updating rewrite-cljc to use the new cli syntax, and updating from -A -m to -M -m is not causing me any terrible discomfort. simple_smile#2020-10-0122:28markbastianOut of curiosity, what is the purpose of the sourceDirectory tag addition in the pom target? I don't see where it is used by other mvn tasks and the clojure-maven-plugin puts the same data into its own config (which I realize is unrelated to tools.deps, but it's the only mvn plugin I can think of that is directly related to building Clojure).
For example:
<build>
<plugins>
<plugin>
<groupId>com.theoryinpractise</groupId>
<artifactId>clojure-maven-plugin</artifactId>
<version>1.8.4</version>
<extensions>true</extensions>
<configuration>
<sourceDirectories>
<sourceDirectory>src</sourceDirectory>
</sourceDirectories>
</configuration>
</plugin>
</plugins>
<!-- How does maven use this? -->
<sourceDirectory>src</sourceDirectory>
</build>#2020-10-0122:33pmonksPretty sure Maven uses that to know where to go to build Java source. IOW it’s probably not relevant for a deployed artifact.#2020-10-0122:35markbastianAh, that makes sense - mixed language project.#2020-10-0517:13zetafishHi all, I try to create an uberjar with depstar using the -X method. I.e. i updated my deps.edn to have an alias specifying a exec-fn and exec-arg. When running on my laptop clojure -X:uberjar a jar is generated just fine. However when I try to create the jar from a docker container it fails with
Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
-X:uberjar (No such file or directory)
The docker images I tried:
clojure:openjdk-11-tools-deps-1.10.1.469
clojure:openjdk-11-tools-deps-1.10.1.561
#2020-10-0518:17kirill.salykinwhat do you think about https://github.com/juxt/pack.alpha?#2020-10-0518:18kirill.salykinit has interesting feature skinny jar#2020-10-0519:54zetafishI want a self contained jar that I can run with java -jar app.jar so skinny does not fit the bill.#2020-10-0517:20alexmilleryou don't have the newest clj tools in your docker#2020-10-0517:21alexmilleryou need clojure 1.10.1.697#2020-10-0519:52zetafishThank you Alex. I presume the 697 image will become available on docker hub soon. In the meantime I built the image locally and the -X:uberjar works fine then.#2020-10-0520:01alexmillerI don't have anything to do with the docker stuff, so no control over that for me#2020-10-0517:57seancorfield@zetafish I'm planning an overhaul of my project's readme files this week to make that clearer (that some of the usage examples require 1.10.1.697). I also need to update all of the clj-new project stuff to generate projects that can work both ways.#2020-10-0518:19dominicmIs clojure.libfile still present in 697?#2020-10-0518:27seancorfieldI don't see it in the system properties when running code via 1.10.1.697 @dominicm -- there's clojure.basis now, which is the path to the combined EDN file.#2020-10-0518:28seancorfieldI see it was still present in 590:
> /usr/local/Cellar/clojure\@1.10.1.590/1.10.1.590/bin/clojure
Clojure 1.10.1
user=> (System/getProperty "clojure.libfile")
"/Users/sean/.clojure/.cpcache/2731801087.libs"
user=>
#2020-10-0518:28dominicmAh, that could explain the bug I'm seeing. I thought it was being retained for backwards compatibility.#2020-10-0518:59seancorfield@dominicm Lots of breakage between the previous stable and the current stable 🙂 but if you change (System/getProperty "clojure.libfile") to (some-> (System/getProperty "clojure.basis") (str/replace #"\.basis$" ".libs")) you can still get at the old file (although, as I recall, the basis file now contains the libs list as an element?)#2020-10-0519:01seancorfieldYeah, (some-> (System/getProperty "clojure.basis") (slurp) (edn/read-string) :libs) should be the same data.#2020-10-0519:50alexmilleryeah, I removed it in 697. we've never publicly doc'ed it and I was not aware of anyone using it (other than add-libs which won't work with latest anyways) so I didn't think there was any reason to retain it#2020-10-0519:51dominicmI was using it to find myself on the classpath and remove my paths#2020-10-0520:00alexmillerI'd say separating the tool from the basis is the better long term path for that#2020-10-0520:01alexmillerwe have an ongoing conversation about some details of that right now#2020-10-0520:21dominicmIf I could reliably identify my lib in the classpath, then that'd be fine 🙂. Happy to switch over to the newnew system when that comes out too.#2020-10-0520:21dominicmA github search indicates I'm probably the only user anyway#2020-10-0520:21dominicmWe also use it in our internal tool based on the lib, but I can update that the same way#2020-10-0521:20seancorfieldThe problem with that (getting a file from the command line) is that the desired order for repos such as ours is:
System -> User (happy to exclude with -Srepro) -> repo-master (this is missing) -> Project -> -Sdeps
and because "last one wins" etc for some combinations of aliases, you can't always specify "repo-master` via some sort of -Sdeps-file option.#2020-10-0521:24seancorfieldAlthough, if you could specify an arbitrary list of files to read in order via -Sdeps-file (and they didn't have to be called deps.edn) then we could use -Sdeps-file ../versions/master-deps.edn:project-deps.edn and not have an actual deps.edn file in each subproject in the repo.#2020-10-0521:25borkdude-Sdeps-path might be a more appropriate name and I'd be happy to support it in deps.clj (and change the name). It already does this now but for just one file#2020-10-0521:27borkdude-Sdeps-overrides could also work as a name to indicate that it doesn't read any other files than these#2020-10-0521:29borkdudebut I'm relatively sure all of this can be done using scripting as well:
merge the configs yourself, provide it via -Sdeps and move the local deps.edn aside (hacky)#2020-10-0521:31borkdudealso I'm relatively sure these things are in some row/column in an excel sheet somewhere on Alex's computer :)#2020-10-0521:37seancorfieldOur deps files are pretty big -- too big to go on the command line I think as merged data -- and, as you say, that's a hacky approach.#2020-10-0521:39seancorfield(also, we're not about to switch from the official Clojure CLI to a third party tool at this point, esp when we would need multiple different binaries for different parts of our dev/test/staging pipeline, whereas we can have just one CLI install)#2020-10-0521:41seancorfieldAnd, yeah, I've already brought this up with Alex (which was how I learned that it's similar to something the Datomic Ions users have asked for) so, yeah, I'll bet it's already in a spreadsheet. I don't know what I'd prefer in terms of a solution. As long as it addresses the deps ordering shown above and is officially supported, I'll be happy with whatever Cognitect/Nubank think is the best approach 🙂#2020-10-0521:43borkdude@seancorfield small details: deps.clj also runs using clojure / JVM as a dependency but yeah I don't expect you to switch - I was just thinking out loud about this option. I'm blathering too much.#2020-10-0522:01seancorfieldAh, true, I'd forgotten there was a library version as well...#2020-10-0522:01borkdudeI just pushed 0.0.10 - there was something with the clojars deploy token, missed that. But now it works:
/tmp $ clojure -Sdeps '{:deps {borkdude/deps.clj {:mvn/version "0.0.10"}}}' -M -m borkdude.deps -Sdeps-file "$HOME/git/clojurescript/deps.edn" -Spath
src/main/clojure:src/main/cljs:resources:/Users/borkdude/.m2/repository/com/cognitect/transit-clj/0.8.309/transit-clj-0.8.309.jar:/Users/borkdude/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20180805/closure-compiler-unshaded-v20180805.jar:...#2020-10-0522:02borkdude(so I'm in /tmp, but I'm using deps.edn from clojurescript in $HOME/git/clojurescript)#2020-10-0523:47alexmillerJust putting this here for now, but there is a new prerelease clj available - 1.10.1.708. Has a small fix to -X error handling and more importantly an update on how dep expansion works when multiple transitive versions of the same dep are in the tree to fix a case where a dep can be incorrectly omitted.#2020-10-0715:42alexmilleranyone tried this?#2020-10-0716:45seancorfieldI'm about to update things here...#2020-10-0718:52seancorfieldBuilding our uber JARs for our 14 processes seems to produce near-identical results between .697 and .708 -- there are variations from run-to-run based (I assume) on differing timestamps etc but nothing more than 150 bytes on JAR files that range from 30MB to 70MB, so that all looks reasonably sane.#2020-10-0718:53seancorfield(if dependencies were missing we'd have known about it by now on .697 and 150 bytes here and there isn't likely to indicate an entire dependency coming or going)#2020-10-0718:59alexmilleryeah, shouldn't be anything different other than the set of jars selected#2020-10-0523:49alexmillerThe circumstances for the latter case are quite complicated so it's a little challenging for me to categorically say whether your deps are affected (they probably aren't), but if you were were interested in trying it, would be happy to have any feedback before I make it stable.#2020-10-0711:58timofor transitioning to clojure tools I am missing the lein install . How does one install a lib to local repository with clojure tools?#2020-10-0711:59dominicm@timok you don't really need to, you can reference it using :local/root#2020-10-0712:00timo😆 easy as that. thanks @dominicm#2020-10-0712:32practicalli@timok as mentioned local/root is very useful. clojure -X:deps mvn-install is a built-in Clojure CLI alias to deploy a Jar of the project locally in the ~/.m2/repository directory#2020-10-0712:37dharriganI find local/root very useful too. I pull down a 3rd party library, do some changes, and without having to deploy it anywhere, I can reference it in my own project by using the local/root structure to point to its sources.#2020-10-0723:00kennyOn 1.10.1.697, if I try to "https://clojure.org/reference/repl_and_main#_launching_a_script," I get the warning "WARNING: When invoking clojure.main, use -M." Is this correct?#2020-10-0723:16alexmilleryes, there have been changes in the clj arguments in 697#2020-10-0723:16alexmillerthose examples are (now) out of date, will fix#2020-10-0723:16alexmillerforgot they were there!#2020-10-0815:41rickmoynihanDoes anyone know if https://clojure.atlassian.net/browse/TDEPS-163 is likely to be implemented any time soon?#2020-10-0815:44alexmillerprobably not, but I did think more about it - the time when it's read by clj is in tools.deps.alpha, which is the process that builds the classpath. at that point you don't have any deps on the classpath (b/c that's what you're making). so it is more complicated than it initially appeared.#2020-10-0815:45ghadicatch 22#2020-10-0815:45rickmoynihanahh so the data-readers aren’t even available yet?#2020-10-0815:45alexmillerno#2020-10-0815:46alexmillertools.deps could do what's in the ticket, and that would open that up for library use of tools.deps. but it won't help you via clj.#2020-10-0815:48rickmoynihannot sure I understand. Would it not be possible to read it with a :default reader in the first pass, and then re-read it later when you have that info?#2020-10-0815:57alexmillerno? you would need to start a whole new jvm to do that#2020-10-0815:52borkdude@alexmiller With regards to env vars your reasoning was: the less tools have to know, the better it is. Doesn't that also go for reader tags?#2020-10-0815:55borkdudewhen opening up reader tags, one could implement one which reads env vars (or wait, it was resolving your home dir), which brings the same trade-offs maybe#2020-10-0815:57alexmilleryeah#2020-10-0816:00rickmoynihanYeah I guess it makes sense not to do it then, thanks for considering it though 🙇#2020-10-0816:08vlaaadwhat if reading tags is done with tagged-literal?#2020-10-0816:09vlaaadand then basis can be read with proper reader tags when there is a classpath..#2020-10-0816:10vlaaadWhat’s the problem reader tags are supposed to solve though?#2020-10-0816:14rickmoynihangiving a data representation to otherwise un(print|read)able things.#2020-10-0816:24rickmoynihanSo my use case is essentially:
I’m using integrant to define systems of components.
Those systems are typically wired together in a base.edn file on the classpath. That file contains a map of integrant components.
However it would be useful to override some parts (key/values) in the integrant system via the command line, and start the thing with clojure -X
Or in prodution allow a user to specify their own profiles as :exec-args data in the deps.edn file directly.
One of the things you may want to override though is an #ig/ref which basically states this component depends on this other component. My current example is a program that I want to communicate with two different deployments of the same system. One of which is authenticated via an auth0 provider, the other is identical but authenticated via a basic auth provider.
I was hoping tagged reader support in the cli tooling would help with this; but it seems I’ll need to implement something to do this. It’s not too complicated; but it’s something I could use in a lot of projects, so I was hoping I could just leverage the cli tools rather than reimplement it each time; or require a library function to do it.#2020-10-0816:50alexmillerYou can read the deps.edn yourself and have any data you like in the aliases#2020-10-0909:00rickmoynihanI’d be happy to do that, but it would also be useful if the cli tools didn’t choke on it:
{:aliases {:reader {;;:exec-fn foo/bar
:exec-data {:tag #foo [some stuff]}}
}}
$ clj -X:reader
Error building classpath. Error reading edn. No reader function for tag foo (/Users/rick/tmp/deps.edn)
If they could set :default to clojure.core/tagged-literal this way would I think work for me.
Is that what you mean by this?
> A good problem would be to avoid any error like that while computing the classpath from the deps.
> That I would be happy to do#2020-10-0909:01rickmoynihanahh also seen @U47G49KHQ appears to be suggesting this 👍#2020-10-0909:02vlaaadyep, that’s what I suggested#2020-10-0909:03rickmoynihanYeah it’s also what I actually meant here:
https://clojurians.slack.com/archives/C6QH853H8/p1602172111259500#2020-10-0909:04rickmoynihanThough I see now it wasn’t clear#2020-10-0913:03alexmillerI actually made this change yesterday in tools.deps https://github.com/clojure/tools.deps.alpha/commit/17f036429b28537c5bb83878cea1c4322fe21d15#2020-10-0913:04alexmillerOh, I didn’t change it for -X param reading though, I’ll get that too before the next release#2020-10-0913:15vlaaadoh, that’s great!!!#2020-10-0913:29vlaaad#clj/requiring-resolve foo.bar/baz here I come#2020-10-0913:30rickmoynihanAhh fab! Thanks @alexmiller 🙇#2020-10-0913:44vlaaadOr bringing #= to edn with #clj/eval (some-code)#2020-10-0816:51alexmillerA good problem would be to avoid any error like that while computing the classpath from the deps.#2020-10-0816:51alexmillerThat I would be happy to do#2020-10-0816:51vlaaadHmm, agree, supplying non-data args via -X is complicated. My api functions in Reveal are supposed to only accept functions as arguments, but to make them available from clj-exec, I have to accept symbols as well and introduce polymorphism just to satisfy clj-exec: https://github.com/vlaaad/reveal/blob/master/src/vlaaad/reveal.clj#L11#2020-10-0816:52vlaaadIf deps supported tagged literals, I wouldn't have to pollute my api#2020-10-0816:53ghadithere is no straightforward way to do it. I wouldn't gate yourself on deps#2020-10-0817:48dharriganI'm curious - what is the caching strategy used for looking up dependencies - I know that a particular dependency has been updated, it shows on clojars, but referencing it as 'RELEASE' in the deps.edn (my main deps.edn, i.e., $HOME/.clojure/deps.edn) I can see that it it's not being pulled down.#2020-10-0817:48hiredmanuse real versions#2020-10-0817:49borkdude@dharrigan it's a simple string hash on the user deps.edn, args and local deps.edn. Use -Sforce to update#2020-10-0817:50dharriganSure, I could do that - but I see for example in the popular seancorfield deps.edn, a lot of references to RELEASE as well. I mean it's not wrong to use RELEASE, it is supported. So surely, something must eventually attempt to find out if it is the latest release or not?#2020-10-0818:36seancorfield@dharrigan The only reason that some things in there use "RELEASE" is so that folks can "guarantee" a recent version of development tools when they first lift aliases from that file (or use the file whole). @U0NCTKEV8 is correct that this is "wrong" insofar as it breaks CLI caching and as @alexmiller says, this is not officially supported by the CLI/t.d.a so it's certainly not a good idea for anything other than quick access to dev tooling. I stopped using it in all the templates baked into clj-new a while back, although it is still used when looking up templates (on Clojars/Maven) to try to get the most recent stable release of a template -- and boot-new and lein new before it all use that same approach, with the same flaws: you can always use the -V / --version option to fetch a specific version of a template if you want/need. I am planning to make a pass over depstar and clj-new to make -X the favored way to invoke those now, including in the generated app template from clj-new, and at that point I'll also make a pass over my dot-clojure repo and start pinning versions, instead of "RELEASE", to ensure folks get a "recent-enough" version of things that actually support -X. Unfortunately, this sort of change is a big ball of yarn that causes a lot of stuff to need unraveling...#2020-10-0817:50hiredmanit is wrong#2020-10-0817:51borkdudeRELEASE is something that maven supports, but it's not encouraged by deps.edn, it just works because it works in maven. They will deprecate it.#2020-10-0817:51hiredmanRELEASE is a mutable reference#2020-10-0817:51hiredmanbuilding caching on top of that is terrible, and clj relies on caching#2020-10-0817:52dharriganOkay, fairy enough. I will use the version, not release 🙂#2020-10-0817:52hiredmanRELEASE is an easy vs. simple thing where people consistently choose what is easy#2020-10-0817:53dharriganIndeed 🙂 We're all lazy 🙂#2020-10-0817:53dharrigan$deity knows I didn't become a programmer to be active 🙂#2020-10-0817:53alexmillerRELEASE is not officially supported by clj#2020-10-0817:53alexmilleruse at your peril (it breaks the cache)#2020-10-0817:54dharrigantbf, it (this library) was the only one in my deps.edn that has release for the coordinate#2020-10-0817:54alexmillerimo you should only ever use it in -Sdeps on the command line, not in a deps.edn file. I reserve the right to enforce that opinion at an arbitrary future time :)#2020-10-0817:55borkdudeit's on the list for clj-kondo warnings here: https://github.com/borkdude/clj-kondo/issues/945#2020-10-0817:56dharriganSomething else, does deps use the .m2/settings.xml at all?#2020-10-0817:56alexmilleryes, for some things (like server creds, mirrors, proxies)#2020-10-0817:56dharriganright right, good. thanks 🙂#2020-10-0817:56alexmillersee https://clojure.org/reference/deps_and_cli#_maven#2020-10-0817:56hiredmanproxies for fetching maven deps, not for git deps?#2020-10-0817:57alexmilleryes, that is still a todo#2020-10-0817:57alexmillerthere is a ticket for it, iirc#2020-10-0817:57alexmillerhttps://clojure.atlassian.net/browse/TDEPS-124#2020-10-0817:58borkdudedeps.clj has support for http / https proxy settings: https://github.com/borkdude/deps.clj - not sure if that's what you mean
https://github.com/borkdude/deps.clj#proxy-environment-variables#2020-10-0818:01borkdudeI think it works in deps.clj because the proxy settings are also passed to the make-classpath invocation#2020-10-0818:02alexmillerthat would make sense#2020-10-0818:02borkdudehttps://github.com/borkdude/deps.clj/blob/02144768876f68445ed2c04cbf90c0b85b0e7d06/src/borkdude/deps.clj#L393#2020-10-0818:12dpsuttoni can get the version from clojure by running clojure --help but it prints with lots of other text obviously. is there a way to get just the version?#2020-10-0818:13dpsuttonwe have a script that invokes -m and wondering whether its better to omit the -M and get a warning or include it and break usage for people who are on older versions#2020-10-0818:14alexmillerclj -Sdescribe will return edn content that includes a :version key#2020-10-0818:14borkdude@dpsutton
$ clojure -Sdescribe | jet --query :version
"1.10.1.697"
#2020-10-0818:15alexmillernote that the warnings are on stderr so you can redirect stderr to null too if you want to not see those#2020-10-0818:16borkdude$ clojure -Sdescribe | clojure -M -e "(require '[clojure.edn :as edn]) (:version (edn/read *in*))"
"1.10.1.697"
#2020-10-0818:16borkdude$ clojure -Sdescribe | bb '(:version *input*)'
"1.10.1.697"
ok, I'm done now :)#2020-10-0818:16alexmillerheh#2020-10-0818:17alexmillerbut I have all these cat skins lying about...#2020-10-0818:18borkdudek, one more:
$ clojure -Sdescribe | grep :version | cut -d' ' -f2
"1.10.1.697"
#2020-10-0818:25lread@dpsutton, I’ve moved to the new clojure command line syntax for rewrite-cljc. I have its scripts fail with an error message if clojure version < 1.10.1.697. My scripts are all in babashka so the check was easy.#2020-10-0818:30dpsuttonthanks everyone#2020-10-0818:38dharriganw00t. It will certainly help those coming fresh to Clojure and seeing your excellent deps.edn example.#2020-10-0818:42seancorfieldOr the Practicalli example (nicer web site, more descriptive names, much better documentation). I'm considering linking to it from my dot-clojure repo and encouraging new folks to use the Practicalli stuff instead -- so I'm under less pressure to keep dot-clojure maintained and documented 🙂#2020-10-0821:29practicalli@seancorfield I would be honoured to be linked from your repo. Thank you.
The library versions used in the Practicalli aliases are updated at least once a month using the :project/outdated alias and version changes added to the changlog.
Versions are updated sooner if I spot a new version or a library maintainer let's me know.#2020-10-0900:29seancorfieldhttps://github.com/seancorfield/dot-clojure has been updated with caveats about it being my personal deps.edn file and to direct people to your repo if they want something well-documented and well-maintained!#2020-10-0821:39seancorfieldseancorfield/depstar {:mvn/version "1.1.126"} -- https://github.com/seancorfield/depstar/releases/tag/v1.1.126 -- just three small bug fixes; follow-up in #depstar#2020-10-0822:00seancorfieldI was planning to make -X the preferred way of using depstar in that release but I realized that -X:uberjar ends up putting the Clojure CLI's version of tools.deps.alpha into the JAR file -- because depstar assumes the "current classpath" is what you want in the JAR (and it excludes itself, assuming it is the only dependency you don't want in the JAR). -X:jar works just fine (because it excludes JAR files). I'll need to modify depstar to use t.d.a directly to build a classpath from deps.edn files (per the discussion the other day) but that gets complicated if you need a "user" deps.edn file and you're using CLJ_CONFIG with a monorepo -- as we are at work -- so that will have to wait for "depstar 2". Given the discussions around that and t.d.a itself recently in this channel, I'll probably defer that work until t.d.a has better support for that (since CLJ_CONFIG is part of the clojure script, not t.d.a).#2020-10-0822:01seancorfieldBTW, CLJ_CONFIG is documented officially (I thought it wasn't):
User - cross-project configuration (typically tools)
Locations used in this order:
If $CLJ_CONFIG is set, then use $CLJ_CONFIG (explicit override)
https://clojure.org/reference/deps_and_cli#_deps_edn_sources#2020-10-0822:13hugodIs it possible to ignore a project’s deps.edn when invoking clojure cli? It would sometimes be useful when passing -Sdeps, e.g. in a tool that bumps a project’s version, or builds docs.#2020-10-0822:14seancorfield@hugod Yes, that's what :replace-deps is for.#2020-10-0822:15seancorfieldSee https://clojure.org/reference/deps_and_cli#_replace_project_environment_tool#2020-10-0822:15hugodthank you - I was looking for a command line switch#2020-10-0822:16seancorfieldYou can use it in -Sdeps but you'd need an alias I believe.#2020-10-0900:31seancorfieldseancorfield/clj-new {:mvn/version "1.1.226"} -- https://github.com/seancorfield/clj-new/releases/tag/v1.1.226 -- preferred usage is via -X; templates updated to reflect -A/`-M`/`-X` changes in recent Clojure CLI versions; follow-up in #clj-new#2020-10-0919:00seancorfieldSigh... seancorfield/depstar {:mvn/version "1.1.128"} and seancorfield/clj-new {:mvn/version "1.1.228"} are available now to fix the bug I introduced in depstar 1.1.126 in license file handling and for clj-new to use the fixed version of depstar; follow-up in #clj-new and/or #depstar (sorry for anyone who used yesterday's broken versions!)#2020-10-1020:36borkdudeMade a small (proof of concept) tool to rewrite EDN files while preserving formatting, whitespace, comments, and reader ignores.
A few examples that might be handy for deps.edn: https://github.com/borkdude/rewrite-edn#examples#2020-10-1113:59Eamonn SullivanHi all. I have a real newbie question on deps.edn. I have the following to add cloverage to my project.
:cloverage
{:extra-paths ["test"]
:extra-deps {cloverage/cloverage {:mvn/version "RELEASE"}}
:main-opts ["-m" "cloverage.coverage"
"-p" "src"
"-s" "test"
"--fail-threshold" "90"]}
The cloverage docs say I should be able to provide a map of options somewhere, like using :fail-threshold instead of the args. How would I do that?#2020-10-1114:16alexmillerI don’t think this is a deps.edn question, not sure how cloverage works#2020-10-1114:18Eamonn SullivanAh, ok, sorry. I thought there was a key I could set like :options {:fail-threshold 90}. Told you it was a newbie question!#2020-10-1315:50tvaughan$ cat tests.edn
#kaocha/v1
{:plugins [:kaocha.plugin/cloverage
:noyoda.plugin/swap-actual-and-expected]
:cloverage/opts {:fail-threshold 100}}#2020-10-1315:51tvaughan@eamonn.sullivan ☝️#2020-10-1315:52Eamonn SullivanAh, thank you. Perfect!#2020-10-1114:19Eamonn SullivanYou can set project default settings for Cloverage in your project. That's all they mention in the docs and I have no idea that means, either. I'll just use the command-line arguments.#2020-10-1114:20alexmillerYeah, don’t know offhand#2020-10-1114:27lread@eamonn.sullivan, I don’t know have active it is, but there is a #cloverage channel.#2020-10-1117:34practicalli@eamonn.sullivan kaocha test runner can also use cloverage too. I have aliases for both, but I don't think I used options yet. Will give them a try and interested to know how you get on
https://practicalli.github.io/clojure-webapps/#2020-10-1212:18borkdudeJust automatically qualified deps in a large deps.edn file (while preserving comments, etc):
https://github.com/borkdude/rewrite-edn/blob/master/examples/qualify_deps.clj#2020-10-1311:09orestisI’m seeing this in Github actions — when trying to do a -X — presumably some weirdness in Clojure version?
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
955
Could not locate clojure/run/exec__init.class, clojure/run/exec.clj or clojure/run/exec.cljc on classpath.
#2020-10-1311:25delaguardohow you setup clojure-cli for your action?#2020-10-1311:26orestisVia… your action 🙂#2020-10-1311:26orestisclojure -Stree gives me this
org.clojure/clojure 1.10.1
1089
org.clojure/core.specs.alpha 0.2.44
1090
org.clojure/spec.alpha 0.2.176
1091
org.clojure/tools.deps.alpha 0.9.816
#2020-10-1311:27orestisAnd clojure --help gives me
Version: 1.10.1.708
#2020-10-1311:27delaguardocould you change the version of my action?)
to this DeLaGuardo/#2020-10-1311:28delaguardonew jar (exec.jar) was introduced recently as a part of clojurecli installation
now it is fixed in master#2020-10-1311:29delaguardosorry for inconvenience (#2020-10-1311:30orestisoh 🙂#2020-10-1311:30orestisNo worries, I was just about to dive into the source code 🙂#2020-10-1314:57borkdudeScript to update pom.xml version, if that's useful to anyone:
https://github.com/borkdude/babashka/blob/master/examples/set_pom_version.clj#2020-10-1315:14flowthingCool! Guess you could also do mvn versions:set -DnewVersion=1.0.1, too, though?#2020-10-1315:15borkdudeShhh#2020-10-1315:15borkdudeHehe. I didn't know that. ^ @UME7B1EBV#2020-10-1315:16borkdudeI guess the script becomes more useful when you want to actually parse the version and bump it.#2020-10-1315:19flowthingCertainly. You might be able to do that with the Maven Release plugin, too, but I’m not sure.#2020-10-1319:39Søren SjørupThe maven version downloads everything on https://apache.org to do the job for me though 🙂#2020-10-1407:39flowthingSure, Maven downloading the entire internet before doing anything is a time-honored tradition. 🙂 It'll only do that the first time you run the command, though.#2020-10-1315:22alexmillerI think maybe I forgot to post it here, but I did a clj 1.10.1.716 prerelease this past weekend that includes a bump in many dependencies (maven resolver, aws api), and changes to make the edn reading more tolerant of unknown tagged literals in alias data#2020-10-1315:25seancorfieldI'll update my dev/test setup today to try it out, and integrate it into our work repo.#2020-10-1315:25alexmillerthx#2020-10-1319:20seancorfieldSo far so good. Now it's on our QA infrastructure and handling deps/building JARs all seems to be working just fine.#2020-10-1509:11rickmoynihanahh amazing I’ll make a note to give it a try at some point. Thanks 🙇#2020-10-1403:10zaneCan anyone think of a reason why someone might be getting USERAUTH fail when using clj to attempt to fetch a public repository via https? 😕#2020-10-1403:22alexmillerIf you could share a repro that would help#2020-10-1403:24zaneSure thing.
clj -Sdeps '{:deps {probcomp/metaprob {:git/url "" :sha "8dc9d09f747c1e29886bb9628a0110c6f6bc6f5a"}}}'#2020-10-1403:24zaneWorks for me. Fails for my colleague.#2020-10-1403:24zaneI’ll get you the exact message.#2020-10-1403:25zaneHere’s what he’s seeing: https://a.cl.ly/llu2n0yR#2020-10-1403:26zaneI’m confused as to why it appears to be attempting to fetch via git ssh.#2020-10-1403:29alexmillerwell it's a git dep#2020-10-1403:29seancorfieldFWIW, that command-line works fine for me on WSL2 (Ubuntu) on Windows 10 and also on macOS 10.12.#2020-10-1403:31seancorfieldCan't test it on Powershell since I don't have the Clojure CLI installed on Windows natively any more.#2020-10-1403:32alexmillerI agree though that it does look like it's trying to use ssh though#2020-10-1403:32seancorfield@alexmiller Where could the jgit stuff be getting auth from that might cause a failure with a public git repo?#2020-10-1403:32alexmillerit's not that that it's not getting auth but that it's looking for it in the first place#2020-10-1403:50zane> well it’s a git dep
Sorry, what I meant was: I’m confused that it appears to be trying to fetch via SSH.#2020-10-1403:51zanePerhaps I’m reading this output wrong?#2020-10-1403:52alexmilleris there any chance that they have a ~/.gitconfig with a url.<base>.insteadOf set?#2020-10-1403:53alexmillerthat allows you to direct git to use a different url instead of the original one and can be used to swap https for ssh urls#2020-10-1403:53seancorfieldOr perhaps they have a deps.edn file in that directory that already specifies that library via {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "b2d5dbc6f2d5dbc6dac7d0"}, :content ("[email protected]")}?#2020-10-1403:54alexmillerthe -Sdeps is merged last so that shouldn't affect anything#2020-10-1403:54seancorfield(they're running that command in their downloads folder so who knows what is in there)#2020-10-1403:55zaneYeah, it’s possible.#2020-10-1403:55zaneI’ll ask.#2020-10-1403:55alexmillershould look for [url " and then an insteadOf line in /.gitconfig - something like that#2020-10-1403:56zaneGot it.#2020-10-1404:00alexmiller[url "
#2020-10-1404:00alexmillerI was able to repro with that#2020-10-1404:00zaneSeems like he’s signed off. I’ll check back in when I can get ahold of him again.#2020-10-1404:00zaneI see! Hopefully that’s the culprit.#2020-10-1404:00zaneThanks for the help!#2020-10-1404:01alexmillercould even be more generic like:#2020-10-1404:01alexmiller[url "git://"]
insteadOf = https://#2020-10-1404:01alexmillercan't say I've ever messed with that config before!#2020-10-1404:02zaneYeah, me either.#2020-10-1404:03zaneThe colleague having the issue is wanting to do some volunteer work on some of our private repositories, and I suspect he probably has a lot going on in his ~/.ssh/config from his job.#2020-10-1404:05zaneFrom talking to Ghadi earlier I know how to set up a new machine for use with clj + private repositories, but I’m not clear yet on how to configure a machine that has a lot of extant ~/.ssh/config settings.#2020-10-1404:06alexmillerwell, the answer is probably ... it depends#2020-10-1404:07zaneFor instance, I know that because of JSch clj doesn’t play well with IdentityFile declarations, and should instead be set up to read keys from the ssh-agent, but I don’t think I can ask this person to change their ~/.ssh/config in a way that might impact their work setup.#2020-10-1404:08zane> it depends
Yeah, I suppose there’s no way out but through. I’m going to have to understand what their existing ~/.ssh/config settings are doing in detail.#2020-10-1418:36zaneTo close this loop, he had this in his ~/.gitconfig:
[url "
#2020-10-1418:37zaneThanks, @alexmiller and @U04V70XH6!#2020-10-1418:37seancorfieldWas he OK with removing that to get clj to work?#2020-10-1418:38seancorfield(I'm kind of curious as to why anyone would want all lein install with deps.edn?#2020-10-1419:29seancorfieldclojure -X:deps mvn-install#2020-10-1419:29seancorfieldhttps://clojure.org/reference/deps_and_cli#_local_maven_install#2020-10-1419:29seancorfield(you'll probably need to generate a pom.xml file first -- I can't remember)#2020-10-1420:07alexmillerno, you don't - you can either rely on the pom inside the jar or supply the g/a/v externally#2020-10-1420:07alexmilleror supply the pom#2020-10-1419:29seancorfieldOh, and that's assuming you're on 1.10.1.697 or later.#2020-10-1419:33dominicm@alexmiller how safe is it to rely on tdeps putting src/main/clojure on the classpath indefinitely?#2020-10-1419:34seancorfield@dominicm I didn't think it did? :paths ["src"] is in the system deps#2020-10-1419:36dominicmOops, via pom.xml*#2020-10-1420:06alexmillerI'm confused by what you mean#2020-10-1420:21dominicm@alexmiller there's always src/main/clojure added when doing a local/root to a pom.xml right now.#2020-10-1421:46alexmillerSo local dep with pom.xml#2020-10-1421:46alexmillerNot planning to change that#2020-10-1501:07Jacob O'BryantThe new git-resolve-tags command appears to have a bug. Someone reported this to me today and I've been able to reproduce as well:
$ cat deps.edn
{:deps
{github-jacobobryant/biff
{:git/url ""
:tag "HEAD"}}}
$ clj -X:deps git-resolve-tags
Error building classpath. Library github-jacobobryant/biff has :tag but no :sha.
Add :sha or run `clj -Sresolve-tags` to update deps.edn.
$ clj -Sresolve-tags
Option changed, use: clj -X:deps git-resolve-tags
$ clj --help | head -n 1
Version: 1.10.1.697#2020-10-2021:24alexmillerfix for this will be in the next prerelease version, coming shortly#2020-10-2021:37alexmilleravailable in 1.10.1.723#2020-10-1503:03alexmillerheh, thanks for the report#2020-10-1900:27athomasoriginalIs there an equivalent of :local/root in the alternative tools e.g. lein or boot?
The reason for the question is because this makes debugging 3rd party library (say reagent) wildly easy. Just clone the lib, add it as a :local/root and then start editing the library as if it was part of your project. Seems like reason enough for Clojure Tools to be used over alternatives.#2020-10-1900:35seancorfield@tkjone I think that is what checkouts is for in Leiningen? https://github.com/technomancy/leiningen/blob/stable/doc/TUTORIAL.md#checkout-dependencies (I've never used that but it sounds a bit like using :local/root in deps.edn)#2020-10-1900:36seancorfieldThere's also a plugin for Leiningen that lets you specify your dependencies via a deps.edn which, presumably, would also serve you for this purposes I think...#2020-10-1900:37seancorfieldhttps://github.com/RickMoynihan/lein-tools-deps -- yes, it supports :local/root and :git/url, according to its README.#2020-10-1900:39seancorfield(`checkouts` would require that the checked-out library be a Leiningen project as well -- Rick's plugin should work for non-lein projects, but for transitive dependencies you'd need it to have pom.xml or deps.edn in it I think)#2020-10-1921:48grzmWas there a change recently (past month or so) in clojure.tools.deps that changed the order of the classpath? In clojure 1.10.1.507 I’m seeing clojure -Spath puts deps.edn :paths first (e.g., with {:paths ["src/main/clojure"] ,,,}, src/main/clojure is the first item in the classpath), and in clojure 1.10.1.645 I’m seeing :paths listed last.#2020-10-1921:50dpsutton> 0.9.799 on Sep 16, 2020- Fix classpath ordering for paths and libs
#2020-10-1921:51grzmCheers! Is there a quick way to correlate tools.deps versions other than looking at the change log?#2020-10-1921:52dpsuttoni'm not sure. i just remembered that change being discussed vaguely. sorry#2020-10-1921:52dpsuttoncorrelate versions to what?#2020-10-1921:52grzmNo worries, you already did more than I could do on my own 🙂#2020-10-1921:53grzmThe clojure-tools release and the clojure.tools.deps releases. Looking at the clojure -h on an installation, I see the former. I’d like to get at the latter more easily.#2020-10-1922:26alexmillerhttps://clojure.org/releases/tools#2020-10-1922:27alexmillerThat’s kind of a consolidated release page for clj and tools.deps (versions of that included for reference) only for stable versions of clj#2020-10-2018:49grzmYeah, found that. I was hoping to have something similar to the output of clojure -h on the command line, to aid in debugging rather than referring to a webpage. That works, though.#2020-10-2018:54alexmilleryou can tell by doing clj -A:deps -Stree | grep tools.deps.alpha#2020-10-2019:54grzmCheers!#2020-10-2019:55grzmwrites that down#2020-10-2000:22cch1I’m getting up to speed with the deploy process using tools from the greater tools.deps ecosystem and I ran into a strange issue today. Essentially, I was finding that libs installed locally did not bring their transitive dependencies along with them. I used clojure.tools.cli.api/mvn-install to do the install.#2020-10-2000:24cch1A bit of investigation shows that mvn-install does not copy the pom file to the local repo -interestingly @seancorfield has observed that slipset/deps-deploy does install the pom file. So is that a bug or a limitation/design decision in c.t.d. mvn-install?#2020-10-2000:24cch1@alexmiller ^#2020-10-2000:32seancorfield@alexmiller I confirmed with @cch1 that running clojure -X:deps mvn-install omits the .pom file and when trying to use that dependency in a clean folder, t.d.a. reaches out to Clojars to look for the "missing" .pom file. When I re-ran the same scenario but using slipset/deps-deploy for the local install, it did copy pom.xml to <library>.pom in ~/.m2 and when trying to use that in a clean folder, t.d.a. did not reach out to Clojars to look for the .pom file. Definitely feels like a bug in t.d.a's mvn-install to me.#2020-10-2000:43alexmillercan you back up to the full command you're running?#2020-10-2000:44alexmillerare you supplying a pom? using the one in a jar file? supplying additional attributes?#2020-10-2000:45seancorfieldThere is a pom.xml in the same directory. I ran just clojure -X:deps mvn-install :jar '"the-lib.jar"' -- do we need extra args to also copy the pom.xml file? It reads the pom.xml file to get the group/artifact/version data...#2020-10-2000:46seancorfieldThe same pom.xml file is inside the JAR as well (so that's the one it reads, based on the reference docs).#2020-10-2000:48seancorfieldAh, if I say :pom '"pom.xml"' as well, then it does copy that to <library>.pom -- is that an intentional difference @alexmiller?#2020-10-2000:53seancorfieldHere's a full repro session:
(! 692)-> rm -rf ~/.m2/repository/seancorfield/
(! 693)-> jar tvf next-jdbc.jar |fgrep pom
110 Mon Oct 19 11:24:06 PDT 2020 META-INF/maven/seancorfield/next.jdbc/pom.properties
1996 Mon Oct 19 11:24:06 PDT 2020 META-INF/maven/seancorfield/next.jdbc/pom.xml
(! 694)-> clojure -Sforce -X:deps mvn-install :jar '"next-jdbc.jar"'
Installing next-jdbc.jar
Installed to /Users/sean/.m2/repository/seancorfield/next.jdbc/1.1.610
(! 695)-> ls -lR ~/.m2/repository/seancorfield/
total 0
drwxr-xr-x 4 sean staff 136 Oct 19 17:51 next.jdbc
/Users/sean/.m2/repository/seancorfield//next.jdbc:
total 8
drwxr-xr-x 4 sean staff 136 Oct 19 17:51 1.1.610
-rw-r--r-- 1 sean staff 305 Oct 19 17:51 maven-metadata-local.xml
/Users/sean/.m2/repository/seancorfield//next.jdbc/1.1.610:
total 96
-rw-r--r-- 1 sean staff 164 Oct 19 17:51 _remote.repositories
-rw-r--r-- 1 sean staff 44300 Oct 19 11:24 next.jdbc-1.1.610.jar
(! 696)-> rm -rf ~/.m2/repository/seancorfield/
(! 697)-> clojure -Sforce -X:deps mvn-install :jar '"next-jdbc.jar"' :pom '"pom.xml"'
Installing next-jdbc.jar and pom.xml
Installed to /Users/sean/.m2/repository/seancorfield/next.jdbc/1.1.610
(! 698)-> ls -lR ~/.m2/repository/seancorfield/
total 0
drwxr-xr-x 4 sean staff 136 Oct 19 17:52 next.jdbc
/Users/sean/.m2/repository/seancorfield//next.jdbc:
total 8
drwxr-xr-x 5 sean staff 170 Oct 19 17:52 1.1.610
-rw-r--r-- 1 sean staff 305 Oct 19 17:52 maven-metadata-local.xml
/Users/sean/.m2/repository/seancorfield//next.jdbc/1.1.610:
total 104
-rw-r--r-- 1 sean staff 188 Oct 19 17:52 _remote.repositories
-rw-r--r-- 1 sean staff 44300 Oct 19 11:24 next.jdbc-1.1.610.jar
-rw-r--r-- 1 sean staff 1996 Oct 19 11:16 next.jdbc-1.1.610.pom
(! 699)-> ls -l pom.xml
-rw-r--r-- 1 sean staff 1996 Oct 19 11:16 pom.xml
#2020-10-2000:54seancorfieldI see Installing next-jdbc.jar in the first case and Installing next-jdbc.jar and pom.xml in the second case so I suppose it is telling you what it is doing...#2020-10-2000:56seancorfieldWithout the .pom file in the repo, attempts to use the dependency will still reach out to Clojars/Maven for that .pom file and if it isn't found, it doesn't look like the pom.xml inside the JAR is sufficient for t.d.a to pick up the dependencies.#2020-10-2000:59alexmillerit will only use the pom if you pass it explicitly (doc string: https://clojure.github.io/tools.deps.alpha/clojure.tools.cli.api-api.html#clojure.tools.cli.api/mvn-install)#2020-10-2001:01alexmillerreaching out to maven is surprising though, so I'll look at that tomorrow#2020-10-2001:03seancorfieldDownloading: seancorfield/next.jdbc/1.1.610/next.jdbc-1.1.610.pom from clojars
^ That's what I get trying to use the installed dependency without the .pom file in the repo. If I install the pom.xml file as well (or use slipset/deps-deploy install) so that I get the .pom file in the repo, that line does not appear when trying to use the dependency.#2020-10-2001:03seancorfieldFeel free to DM me tomorrow if you need me to run more test cases.#2020-10-2001:03seancorfield(and there was no attempt to download the .jar file, only the .pom file)#2020-10-2001:04alexmillersorry, maven looking for the pom file is normal behavior, I thought you were saying it was reaching out for the pom file when installing#2020-10-2001:05seancorfieldRight, but that will fail if there is only a local install -- and the problem Chris was seeing was that if no .pom file existed, transitive dependencies were not being found.#2020-10-2001:06alexmillerwell that's to be expected#2020-10-2001:06alexmillerthe question is what you supply on install#2020-10-2001:06alexmilleryou have to either supply a pom, or use the pom in the jar, or is this the case where you did neither?#2020-10-2001:07seancorfieldThere is a valid pom.xml inside the JAR -- I showed that above.#2020-10-2001:07alexmillersorry, too much reading, and I'm a drink into the night :)#2020-10-2001:07seancorfieldThe problem is that the .pom file is not created in the local repo on the install. Unless you explicitly specify :pom '"pom.xml"'#2020-10-2001:08alexmillerok, that's surprising to me, should be using the one in the jar in that case#2020-10-2001:08seancorfieldOK, we can follow-up tomorrow. Feel free to DM me about it.#2020-10-2001:08seancorfieldSee my console session above.#2020-10-2001:23alexmilleryeah, that code is not there. it is using the pom in the jar to pull G/A/V but it's not actually installing it as a distinct artifact. not sure why I thought that would happen automatically, maybe tricked myself with some caching or something#2020-10-2001:23seancorfieldSo, definitely a bug? Want me to create a JIRA?#2020-10-2001:24alexmillerdefinitely a bug, sure on jira#2020-10-2001:24alexmillershould be easy to fix#2020-10-2001:28seancorfieldhttps://clojure.atlassian.net/browse/TDEPS-169#2020-10-2001:42seancorfieldseancorfield/depstar {:mvn/version "1.1.132"} -- calls shutdown-agents at the end to account for "badly-behaved" code being AOT'd that has side-effecting top-level forms that start threads (e.g., Neanderthal) -- follow-up in #depstar#2020-10-2001:52cch1Sorry I’m late catching up here. I don’t have much more to offer than what Sean noted above. Let me know if I can help test.#2020-10-2021:36alexmillerClojure tools 1.10.1.723 (a prerelease) is now available:
• Fix clj -X:deps tree adding tools.deps.alpha to tree
• Fix clj -X:deps mvn-pom adding tools.deps.alpha to pom deps
• Fix clj -X:deps git-resolve-tags not working
• Fix clj -X:deps mvn-install on jar to also install embedded pom
#2020-10-2021:47alexmillerthe first 3 were actually all a bug in the default :deps alias using :extra-deps rather than :replace-deps (duh)#2020-10-2114:03alexmillerif anyone tries that prerelease and has good/bad feedback, please let me know, I will probably release it as stable by tomorrow#2020-10-2116:35seancorfieldI updated our repo at work to use it immediately but I haven't exercised it much yet (and in particular I haven't exercised the options you fixed yet!). I'm just about to update our "ancient" script which uses -Stree to use -X:deps tree instead and see if it still works correctly...#2020-10-2116:36seancorfieldOh, I misspoke. Apparently I made that change a while back so I have already tested that it didn't break anything...#2020-10-2116:37seancorfield...and I can confirm that all the maven-resolver-* deps disappeared, as expected. So, yup, that worked.#2020-10-2116:44seancorfieldOK, I just ran clojure -Spom across a dozen projects and then clojure -X:deps mvn-pom and confirmed there's no differences between those two now.#2020-10-2116:49alexmillerthere is still a difference in that your command line classpath-modifying arguments won't change the generated pom with the latter#2020-10-2116:50seancorfieldRight, I don't rely on that in any of the projects where I create pom.xml files at the moment.#2020-10-2116:51seancorfieldBut, yeah, we couldn't use this in our monorepo.#2020-10-2116:52alexmillerstill a TBD but I think we will probably have some semi-standard set of args that you can use to do that on mvn-pom if desired (or in tree). not sure yet whether that is args that mirror clj switches or pass through of some kind that gets parsed etc.#2020-10-2116:56seancorfieldThat did cause me to notice that
CLJ_CONFIG=../versions clojure -A:defaults -Spom
and
CLJ_CONFIG=../versions clojure -R:defaults -Spom
behave differently. The latter works, in that it merges the :defaults alias (with all our :override-deps) into the lib map which -Spom sees but the former fails to do that.#2020-10-2116:58seancorfieldWith -A:defaults I get this error
Error generating pom manifest: No coordinate type found for library compojure/compojure in coordinate {}
because that's how we specify common deps that have :override-deps in the :defaults alias in our "master" deps.edn file.#2020-10-2118:21alexmillerI can see why that is - the gen script piggybacks the normal cp script and one of the arg names changed in the latter so it's getting missed#2020-10-2118:21alexmillerat the time that got changed, the gen script had been removed so didn't show up in the refectoring but was later re-added#2020-10-2118:38alexmiller@seancorfield fixed in 1.10.1.727#2020-10-2118:39seancorfieldI will update to that shortly 🙂#2020-10-2119:21seancorfieldUpdated. Confirmed fixed. Thanks @alexmiller#2020-10-2210:05practicalliShould the -M flag for Clojure CLI tools have an issue when also supplying -Sdeps options? (the -A flag works without issue for the same command)
/usr/local/bin/clojure -M:env/test -Sdeps '{:deps {}}'
This gives an error, whether or not the :deps contains any dependencies.
Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
-Sdeps (No such file or directory)
I first experienced this issue when running Clojure CLI tools with Cider (Emacs) and using the -M flag, specifically with cider-jack-in which injects the relevant dependencies for cider-nrepl using `-Sdeps argument. I can disable the actual dependencies, but not yet
Testing the same command on the terminal, gives the same error.
Using the command without the -M:env/test alias, only the -deps arguments works
Using the -A flag works with S-deps argument
Starting a repl with that command, minus the -Sdeps argument works via the command line and I can connect
I have tried this with several versions of Clojure CLI tools installed, from 1.10.1.697 onwards, including the latest pre-release. All seem to have the same issue.#2020-10-2210:14borkdude@jr0cket The -S... options should go before -M#2020-10-2210:16borkdudeclj-opts before exec-opts (although they are mentioned in the other order in the help text, that order is documented as Run main clojure [clj-opt*] -M[:aliases] [init-opt*] [main-opt] [arg*] )#2020-10-2210:19borkdude(I've made the same mistake before)#2020-10-2210:20practicalliYes, putting the -M after -Sdeps does seem to work on the terminal.
That ordering is not the case with -A flag and so sounds like a change is required for Cider to work correctly with jack-in. The ordering can be changed by editing the the command line each time, but using .dir-locals.el config to set the alias, the -M and -A flags are put before the -Sdeps option
I assume -A will also work after -Sdeps option, to support those who havent upgraded to Clojure CLI tools#2020-10-2210:21borkdudethe same rules apply for -A#2020-10-2210:22borkdudeStart a REPL clj [clj-opt*] [-A:aliases] [init-opt*]
#2020-10-2210:23practicalliExcept -A doesnt uphold that rule in practice...#2020-10-2210:23borkdudeIt's a case of Hyrum's law, but I guess the tools doing it wrong should be changed.#2020-10-2210:27practicalliConfirmed that editing the cider command line and putting -M flag in the correct place does work :)
/usr/local/bin/clojure -Sdeps '{:deps {nrepl {:mvn/version "0.8.2"} refactor-nrepl {:mvn/version "2.5.0"} cider/cider-nrepl {:mvn/version "0.25.3"}}}' -M:env/test -m nrepl.cmdline --middleware '["refactor-nrepl.middleware/wrap-refactor", "cider.nrepl/cider-middleware"]'
I'll raise an issue with Cider to see if they can fix this. Thanks.#2020-10-2211:47dominicmThat's annoying. It looks like we'll need to write a parser for the arguments or something and split on it? Not too terrible, but it is if you want to be backwards compatible.#2020-10-2211:50practicalli@dominicm I just submitted an issue for this on the cider repository
https://github.com/clojure-emacs/cider/issues/2916#2020-10-2311:10practicalliAnd there is a pull request that fixes this issue https://github.com/clojure-emacs/cider/pull/2917#2020-10-2211:51dominicmHmm, I guess I'll have to check if my scripts use prefix or suffix form. That pattern is used in other tools too.#2020-10-2212:09borkdudeor just don't use cider-jack-in. I never use it.#2020-10-2213:22dominicmI do. It's a useful tool, especially for beginners. Having less pieces to understand is very useful.#2020-10-2213:24borkdudeHaving less magic to understand is also useful ;)#2020-10-2213:26dpsuttonI also really like that CIDER can start up almost any project for you automatically without thought. It displays its startup command in the repl so you know there actually isn’t any magic. It’s just an extra dep and a main so you can run from the clj easily#2020-10-2213:54dominicmThere's no magic, just trying to build on top of simple tools underneath. It's more like an abstraction.#2020-10-2213:54borkdudeIt feels like too much magic to me. Also I don't want the REPL to be a subprocess of my editor..#2020-10-2217:33dominicmimplementation detail, subproc part could change :)#2020-10-2213:55dpsuttonthe subprocess part is fair 🙂 If there's a problem in your lisp interpreter from 1970 your repl goes down ...#2020-10-2214:12alexmillerClojure tools (clj) https://clojure.org/releases/tools#v1.10.1.727 is now available:
• Fix clj -X:deps tree adding tools.deps.alpha to tree
• Fix clj -X:deps mvn-pom adding tools.deps.alpha to pom deps
• Fix clj -X:deps git-resolve-tags not working
• https://clojure.atlassian.net/browse/TDEPS-152 - Fix clj -X:deps mvn-install on jar to also install embedded pom
• Fix clj -Spom not respecting dep modifications from -A (regression)
#2020-10-2217:32rschmuklerIs there any way to install dependencies from a given alias without running an associated :main-opts in the alias? Essentially I want a -e "" that ignores the main-opts#2020-10-2217:33rschmukler(specifically from the CLI, I realize its very doable via the clj package)#2020-10-2217:42rschmuklerRight now my best shot is using babashka to construct a new deps.edn and invoking clj -Sdeps
clj -Sdeps "`cat deps.edn | bb -I -e "{:deps (merge (:deps (first *input*)) (get-in (first *input*) [:aliases :native-image :extra-deps]))}"`" -e ""
But this feels like quite the hack#2020-10-2217:48borkdude@rschmukler (I don't think you need -I here, since you're reading only one EDN value)#2020-10-2217:49borkdudecat deps.edn | bb -e "{:deps (merge (:deps *input*) (get-in *input* [:aliases :native-image :extra-deps]))}"#2020-10-2217:50rschmuklerRight on! I misunderstood the CLI. I thought -I coerced to EDN from string literal#2020-10-2217:50borkdude-I is for reading multiple EDN values from stdin#2020-10-2217:50seancorfieldThe official way is to use the -P option: https://clojure.org/reference/deps_and_cli#_prepare_for_execution#2020-10-2217:52seancorfieldSo clojure -P -M:my:aliases:here assuming you're using a recent release of the CLI.#2020-10-2217:52rschmukler@seancorfield perfection! Sorry that I missed that. Might be worth adding that to the clj --help docs too#2020-10-2217:53seancorfieldIt's in that help
exec-opts:
-A:aliases Use aliases to modify classpath
-X[:aliases] Use aliases to modify classpath or supply exec fn/args
-M[:aliases] Use aliases to modify classpath or supply main opts
-P Prepare deps - download libs, cache classpath, but don't exec
#2020-10-2217:53rschmuklerThen clearly my clj must be outdated lol!#2020-10-2217:53rschmuklerRight'o - thank you for the help!#2020-10-2217:53seancorfield(! 652)-> clojure -Sdescribe
{:version "1.10.1.727"
:config-files ["/usr/local/Cellar/#2020-10-2217:54rschmukler:version "1.10.1.645
#2020-10-2217:54rschmuklerThere we are#2020-10-2217:56seancorfieldYeah, 645 was one of the prerelease builds while a lot was changing in the CLI.#2020-10-2217:59rschmuklerNow I've just gotta figure out why Manjaro hasn't kept up#2020-10-2218:08rschmuklerUpdated my clojure and sure enough, it's all there for me. Thanks again @seancorfield#2020-10-2218:18practicalliClojure CLI 1.10.1.697 is the minimum version for the -P and new mode for the -M flags. 1.10.1.727 is the latest (officially release today)#2020-10-2218:19alexmillerhttps://clojure.org/releases/tools for your version questions#2020-10-2218:19seancorfieldI'll be glad when -A changes to no longer run :main-opts 🙂#2020-10-2218:26seancorfield@U064X3EF3 This is still the plan, at some point, right?#2020-10-2218:31alexmilleryes#2020-10-2218:32alexmillerI expect that to be ~months#2020-10-2218:21practicalliI did a little cheat sheet of command line flags to help me remember https://github.com/practicalli/clojure-deps-edn#clojure-cli-main-flag-options#2020-10-2222:11seancorfield@jr0cket I see
Run the project clojure -X:project/run -m domain.main-namespace
but there's no :project/run alias in your deps.edn so I'm not sure what this is meant to do? (and it seems weird to have -X and then have -m domain.main-namespace as well?)#2020-10-2222:12seancorfieldAlso, you have
Package library clojure -X:project/jars
That should be -X:project/jar singular, yes?#2020-10-2309:35practicalliThanks for the review, those issues are now fixed, along with further explanation.
Using a :project/run alias seems to make most sense as a project level deps.edn, so the specific namespace and function names are in the config. I'm adding this to the templates I'm creating with clj-new.#2020-10-2222:03drewverleeWhen I run clj i get an exception about building the class path. I tried deleting my cpcache. If you can think of something else i can try please chime in.
gist file is my deps.edn. the comment contains the command run and output (error): https://gist.github.com/drewverlee/893a150c780a86fbf73cd789b48d9b40
Thanks in advance. ill post in this thread if i figure it out.#2020-10-2222:05dpsutton#:com.datomic{ion-dev #:mvn{:version "0.9.276"}} the ion-dev does not begin with a colon but in #:mvn{:version it does. can you try fixing that?#2020-10-2222:10dpsuttonscratch that. namespaced symbols look strange to me in that fashion 🙂#2020-10-2222:12drewverleeIt is strange, I'll clean it up. But I'm guessing that's not it. That's from copy pasting#2020-10-2222:12alexmillerI don't think that's it - this look like maybe a bad version range in some transitive dep?#2020-10-2222:13alexmilleralthough I know all those deps so not sure what you'd find there :)#2020-10-2222:14seancorfield{:deps #:com.datomic{ion-dev #:mvn{:version "0.9.276"}}
under an alias, that should be :extra-deps, not :deps (or :override-deps)#2020-10-2222:15alexmillertypo: mvn/verison#2020-10-2222:15seancorfieldAh yes, on com.datomic/ion {:mvn/verison "0.9.48"}#2020-10-2222:15alexmillernewest version of clj will give you a better error message there#2020-10-2300:08drewverleei see the typo now. ugh. I should turn on a spell check before i post something to catch things like this.#2020-10-2300:42alexmillerOn latest clj, you'll get: Error building classpath. No :mvn/version specified for com.datomic/ion#2020-10-2307:08dominicmNow that add-lib isn't functional, if I wanted to load dependencies for a particular function call with exec-opts (without the user having to know which dependencies to add to deps.edn for that call) how would I do that?
I want to provide some beginner tools as part of something, but I don't want to bloat the primary use case.
If I had bash-style exec, I'd probably just restart the jvm that way.
I want the set of functionality to be an open-set though, so it might expand in the future, and the user shouldn't need to fiddle around to get that working. I'm certainly aiming for "easy" here.#2020-10-2307:24dominicmJust looked at what ion does, that has ion-dev which you install in your user deps.edn, that strikes me as an odd choice.#2020-10-2310:14dominicmI suppose this could be considered the ability to import deps.edn aliases into your project somehow :)#2020-10-2311:17alexmillerPutting ion-dev in user deps.edn is recommended - it’s a general tool for deploying ions#2020-10-2311:20alexmillerWe will be updating add-lib to work again btw with some additional apis and ultimately in master#2020-10-2311:21alexmillerBut not sure I have a good answer for this use case right now#2020-10-2311:46dominicmnp. Maybe I'll start with bloat, then swap to add-lib later.#2020-10-2316:41seancorfieldI built a version locally, using Maven, based on a t.d.a add-lib branch a while back I think. Whatever was the equivalent of 0.8.667 it seems.#2020-10-2316:45seancorfieldI built it on April 6th, so it's probably the head of the add-lib branch, which was 0.8.667 plus the calc-basis stuff, looking at the commit logs. @dominicm I guess keeping your own version around for this isn't going to work for your use case?#2020-10-2316:46dominicm@seancorfield it would yeah, I had assumed that stuff was completely broken by the internal changes. I'll give it a go, thanks!#2020-10-2316:50seancorfield-rw-r--r-- 1 sean admin 58110 Apr 6 2020 tools.deps.alpha-0.8.667-add-lib.jar
In case you want a sanity check on the size of the JAR after you build it. Then I just use :local/root to depend on the JAR.#2020-10-2316:52dominicmI'll probably have to deploy it somewhere, but that's fine :)#2020-10-2319:03alexmillerI just pushed an add-lib3 branch that's up to date with master#2020-10-2319:04alexmillernote that add-lib is now add-libs and there are some other additional apis (not final) in there too - find-revs and find-lib#2020-10-2319:08alexmilleralso, at some point in the past I made it degrade gracefully if it doesn't have the compiled s3 class stuff so it's a little easier to hack onto a classpath like it was originally#2020-10-2319:10dominicmGreat, I guess I know what I'm doing tomorrow then!#2020-10-2319:11dominicmMy goal is that beginners can do (require 'beginner) or something, and it'll go ahead and fetch tools like spyscope and dot-clojure2 for them, and load them up.
Perfect compromise between beginners and advanced users (who will set that stuff up for themselves eventually)#2020-10-2321:29seancorfield@alexmiller Thanks for the add-lib3 branch -- very nice that it works from source again, albeit without S3 support! Just updated my .clojure file for that.#2020-10-2308:30Eamonn SullivanGood morning, does anyone use CircleCI with tools-deps and have an example I can see? I'm just exploring a few alternatives to our Jenkins. I use Github actions on my own stuff (e.g., https://github.com/eamonnsullivan/github-pr-lib I create for scripts and deploy to clojars) and I'm also going to take a look at AWS CodeBuild/Pipeline, but examples to study are mostly lien-based.#2020-10-2308:33borkdude@eamonn.sullivan Almost all of my projects do#2020-10-2308:34Eamonn SullivanAh, perfect. Thank you!#2020-10-2308:34borkdudehttps://github.com/borkdude/clj-kondo/blob/master/.circleci/config.yml#2020-10-2309:16practicalli@eamonn.sullivan There are images you can choose from that already have Clojure CLI tools included.
https://practicalli.github.io/clojure/testing/integration-testing/circle-ci/
If you want the very latest version of Clojure CLI tools, then use Borkdude's approach (which I am going to add to that page I mentioned right now, as I want to use 1.10.1.727)#2020-10-2309:46practicalli@eamonn.sullivan I have a collection of different approaches I'm going through in this ticket https://github.com/practicalli/clojure-content/issues/183
Would be interested to hear what you end up using. Thank you.#2020-10-2319:59Eamonn SullivanSo, you asked how I got on: I decided to try one of the pre-built images. The test/pull-request checks are working, but not the deploy. I get a TransferFailedException. I'm literally only hours old as a CircleCI user and I'm probably doing something basically wrong.#2020-10-2713:39Eamonn SullivanSo, just to finish this thread off, in case anyone is still reading: I did eventually get it to work, providing the same functionality that I have (on other repos) using Github Actions, which was the goal. Thanks very much for the help and the pointers! https://github.com/eamonnsullivan/github-api-lib/blob/main/.circleci/config.yml#2020-10-2715:50practicalliThanks for sharing#2020-10-2310:27dominicmHum, I think there's a mistake in exec.clj. https://github.com/clojure/brew-install/blob/f734dacaab43c0cd44c5ffffb8fa3a10ab64d4a0/src/main/clojure/clojure/run/exec.clj#L20 this throws, but almost all calls also throw, https://github.com/clojure/brew-install/blob/f734dacaab43c0cd44c5ffffb8fa3a10ab64d4a0/src/main/clojure/clojure/run/exec.clj#L26#2020-10-2311:23alexmillerYeah that doesn’t look right#2020-10-2311:44dominicmI also noticed that read-args uses a loop but could be mapv quite easily#2020-10-2311:52alexmillerfixed the double throw#2020-10-2312:05alexmilleragreed on read-args - that stuff has been through a lot of versions where the positionality was more important, going to leave it for now as it's reasonably likely to go through more changes#2020-10-2314:21dominicm@alexmiller I'm currently working on a main which will have exec-like uses, particularly the notion of updating a map in response to those args. Might be useful to rely on the exec code once it's stable. Then I can just point at your docs :p#2020-10-2314:23dominicmObviously read-args is pretty simple, and so is the reduce itself. So maybe that's not really useful. But I'm not sure how it'll evolve. The tagged-literal reader might be subtle.#2020-10-2314:23alexmillerI would not rely on the exec code right now - I do not (yet) consider that to be "public api"#2020-10-2314:24alexmillerI think it would actually be useful to move some or all of it into clojure itself (similar to clojure.main) but that's probably longer term#2020-10-2314:27dominicmYeah, I saw it was all private and ducked out :)#2020-10-2311:09arohnerWhat is the recommended way to deploy a jar to a private S3 repo using tools.deps?#2020-10-2314:08dominicmThis seems like an argument against exec-opts to me: clj -X:user/new create :template "\"$PWD/template::new-app\"" :name my.project.name (this doesn't work for me, but that's unrelated, just the awkward way to pass in pwd)#2020-10-2314:17alexmillerwhat doesn't work?#2020-10-2314:18alexmillerand it's awkward because? quoting?#2020-10-2314:24dominicm@alexmiller yeah, the quoting is awkward, compare to main-style:
--template "$PWD/template::new-app"
Doesn't work because I'm passing wrong args to clj-new, but I didn't need that debugging.#2020-10-2316:50seancorfield@dominicm an FYI on usability there: both clj-new and depstar ended up allowing symbols or strings in most argument slots because of quoting so :template foo works because it is silently converting the symbol to a string behind the scenes. But of course you can't use symbols in all cases (and particularly yours).#2020-10-2316:50dominicm@seancorfield heh, that feels like a dangerous path with spaces and backslashes, etc.#2020-10-2316:51dominicmI wonder if a directory named "4" would work, for example#2020-10-2316:51seancorfieldYup, it only works for "strings" that could be legal Clojure symbols... No, because that would be read as a number, not a symbol.#2020-10-2316:51dominicmEven my example is dangerous because it doesn't escape backslashes.#2020-10-2316:52dominicmSeems like you wouldn't want to script anything on that basis then.#2020-10-2316:52seancorfield#2020-10-2316:56seancorfield(because I just call str on it -- I thought I was being fancier than that)#2020-10-2316:59dominicmStr on a number works I guess?#2020-10-2317:04dominicmAnyway. Not a problem/solution situation really. Just an observation that this decision had this property.
I'm sure eventually we'll know when to apply each decision (or find a new decision) to get desired properties.#2020-10-2317:13seancorfieldYeah, that's why I crossed that out. I tried :4 as one of the args and got the equivalent of ":4" 🙂 Much depends on whether you want a plain string or, ultimately, a qualified symbol or relative path or...#2020-10-2318:36souenzzoI can't find how to do clj -m app.main in "modern #tools-deps#2020-10-2318:38Derek Passenclj -M -m app.main, I believe#2020-10-2318:44borkdudecorrect#2020-10-2319:02alexmillerjust wait till we get to post-modern clj#2020-10-2319:03souenzzocan I stay using clj -m app.main on my deploy/scripts?#2020-10-2319:09alexmillerfor now, yes (eventually that will stop working but not soon)#2020-10-2319:06borkdudein post-modern clj there is no one big story of arguments, but lots of different forks of clj where everyone does their own thing#2020-10-2319:08borkdudeno one believes that hammock time eventually leads to the ultimate truth anymore. instead writing clojure programs becomes an expression of doubt#2020-10-2319:08borkdude:P#2020-10-2319:08dominicmI haven't released stable software since spec-ulation#2020-10-2319:09borkdudeso hasn't core maybe? I should check#2020-10-2319:09alexmiller1.10.1 :)#2020-10-2319:10drewverleeWhen i start an nrepl server through cider and eval datomic.client.api/client it complains about missing hmc auth. Indeed, if i check my cider-classpath that dep is missing.
The issue is the same as discussed here: https://clojurians-log.clojureverse.org/datomic/2020-07-22, which the solution being to update clojure. I updated via linuxbrew and now my version of clojure tools is 1.10.1.727. The issue remains.
My only idea is that if i ask cider to desribe the connection it says clojure is on 1.10.1 not 1.10.1.727 i'm not sure if its truncating or if it somehow connecting to an old version#2020-10-2319:11alexmiller727 is the version of the clojure tools. the actual clojure version really is 1.10.1#2020-10-2319:12alexmiller(unobviously, any version of the clojure tools can use any version of clojure - the clojure tools part really just tells you what it was built with and what version of clojure you get by default if you don't specify one)#2020-10-2319:13dominicm❯ clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.0.0"}}}'
Downloading: org/clojure/clojure/1.0.0/clojure-1.0.0.pom from central
Downloading: org/clojure/clojure/1.0.0/clojure-1.0.0.jar from central
Clojure 1.0.0-
user=> *clojure-version*
{:major 1, :minor 0, :incremental 0, :qualifier ""}
user=>
#2020-10-2319:16dpsuttonpossible the erroneous classpath computed by an older tools deps is cached? just delete the cache?#2020-10-2319:16borkdudeor use -Sforce#2020-10-2319:18drewverleeI deleted .cpcache (which i assume is the only cache) and it didn't work. I tried -Sforce, which starts a repl but then I can't connect to it with cider. I just get a "connection failed".
I'm going to try restarting my computer and assume this is a cider thing. Thanks for the suggestions.#2020-10-2319:19dpsuttonits not CIDER. and restarting the computer won't clear the cache that is suspected to be at issue (if we are correct)#2020-10-2319:21alexmiller.cpcache is the classpath cache so sounds like you've done that already#2020-10-2319:21dpsuttonoh wow. i totally glossed right over that message. sorry#2020-10-2319:22alexmilleroh that hmac auth thing was actually a tools deps bug and should be fixed in latest#2020-10-2319:23dominicmI wonder if there's 2 copies of clojure in $PATH or something...#2020-10-2319:23alexmilleryeah, can you repro outside cider?#2020-10-2319:23alexmillerif not, I'd guess you're somehow seeing a different version of clojure, or something is explicitly using an older version of tools.deps.alpha#2020-10-2319:24dominicmcider literally just runs clojure -Sdeps '{nrepl/nrepl … cider/cider-nrepl …}' -m nrepl.cmdline …#2020-10-2319:25drewverleecider does just run clojure -Sdeps, but ill try using it directly just in case.#2020-10-2319:26dominicmcider's jack-in isn't magic, it's just saving you knowing which dep & which main to start.#2020-10-2319:27dpsutton(and now prints that and optionally the cljs startup to remove any semblance of magic)#2020-10-2319:40drewverleeThe issue was corrected by having cider look at a version of clojure tools that had the bug fix. I had two versions of clojure installed (no useful reason for that). So i uninstalled both and reinstalled using linux brew.
As dominicm points out, you can look at what version of clojure cider is using in the message. It was different from what i got at the command line. So after verifying one version worked but not another, it was fairly obvious.
When i run clj /clojure from the command line it tells me the version of clojure, but in this case it seems it was the version of tools deps that was different between the two.
e.g /usr/local/bin/clojure starts a repl with "clojure 1.10.1" and the other version in /home/drewverlee/.linuxbrew/bin/clj says "clojure 1.10.1" so part of me assumed they were a symlink/alias.#2020-10-2320:02alexmillercool#2020-10-2413:21dominicm{:aliases {:foo ["bar"]}} and clj -A:foo produces interesting results :P
e.g.
:resolve-args {\b \a},
:classpath-args {\b \a},
#2020-10-2413:33dominicmI was curious to see what would happen :)#2020-10-2414:43holyjakHi! What is the relationship of versions between Clojure and tools.deps? Latest Clojure download is https://download.clojure.org/install/linux-install-1.10.1.727.sh, latest https://github.com/clojure/tools.deps.alpha#release-information. If I install that Clojure, which deps will I get? I.e. what command-line options will clojure support? Thank you!#2020-10-2414:44alexmillerThe change logs for the two are cross linked and you can find official release info at https://clojure.org/releases/tools#2020-10-2414:45alexmillerThe docs on the Clojure site always reflect the latest stable release (or use -h or the man page)#2020-10-2414:46alexmillerhttps://clojure.org/reference/deps_and_cli#2020-10-2414:51holyjakThank you! I should have noticed the " See the changelog for version information." at https://clojure.org/guides/getting_started#2020-10-2415:10holyjakQ1: Does :local/root need to be an absolute path? If so - I guess it isn't possible to use something like "~/path/to/it" (to make it portable across users)?#2020-10-2415:13holyjakQ2: Why am I getting
> Error building classpath. No coordinate type found for library cryogen-asciidoc/cryogen-asciidoc in coordinate {:local-root "/Users/me/external/cryogen-all/cryogen-asciidoc"}
when head /Users/me/external/cryogen-all/cryogen-asciidoc/pom.xml returns successfully (modified)
...
<groupId>cryogen-asciidoc</groupId>
<artifactId>cryogen-asciidoc</artifactId>
...
?#2020-10-2415:14dpsuttonhttps://github.com/lambdaisland/chui/blob/master/deps.edn#L4 is an example of relative paths. I think if you use an absolute path like ~/some/path it will not be portable at all#2020-10-2415:18alexmillerYou can’t use ~ - that’s a shell thing#2020-10-2415:19alexmillerBut either absolute or relative are fine#2020-10-2415:21alexmillerNot sure if that solves your q2 or not but if not share your deps.edn#2020-10-2416:20holyjakHere is it
{:deps {org.clojure/clojure {:mvn/version "1.10.1"}
ring-server/ring-server {:mvn/version "0.5.0"}
ring/ring-devel {:mvn/version "1.7.1"}
compojure/compojure {:mvn/version "1.6.1"}
cryogen-core/cryogen-core {:mvn/version "0.3.2"}
#_#_cryogen-core/cryogen-core {:local-root "../../external/cryogen-all/cryogen-core"}
#_#_cryogen-asciidoc/cryogen-asciidoc {:mvn/version "0.3.2"}
cryogen-asciidoc/cryogen-asciidoc {:local-root "../../external/cryogen-all/cryogen-asciidoc"}}
:aliases {;; Run with `clojure -M:build`
:build {:main-opts ["-m" "cryogen.core"]}
;; Start a server serving the blog: `clojure -X:serve`
;; (requires tools-deps 0.9.745+)
:serve {:exec-fn cryogen.server/serve
:exec-args {:port 8888}}}}
This is from https://github.com/holyjak/blog.jakubholy.net, the local dependency is https://github.com/cryogen-project/cryogen-asciidoc/#2020-10-2416:56alexmillerIt’s :local/root not :local-root#2020-10-2417:00holyjakOMG, thanks a lot! I must be blind.#2020-10-2415:36dpsuttonI’m surprised you can’t use ~. I figured it would end up in a path object and the jvm would work it out for you#2020-10-2415:40borkdudejvm doesn't do this#2020-10-2415:40borkdudeit's a bash thing. On Windows this doesn't work either#2020-10-2415:40dpsuttonMakes sense. Not sure where I got that expectation#2020-10-2415:41borkdudeWhat could work OS-independent is interpolation of environment variables. But even then you would have to use different names for linux and windows#2020-10-2415:42borkdudeso probably a script that generates deps.edn for you as a pre-processing step is the least worst way of dealing with such concerns#2020-10-2415:42borkdudeI haven't had an app where I did this, except for one where I keep deps in sync between project.clj and deps.edn#2020-10-2415:50alexmillerJava has a system property for this - user.home#2020-10-2415:50alexmiller(System/getProperty "user.home") is platform independent#2020-10-2415:51alexmillerbut no equivalent to that in deps.edn (yet, or maybe ever)#2020-10-2415:55dominicm@holyjak what's the home-relative use case?#2020-10-2416:13holyjakNone, if I can (I can) use relative psths#2020-10-2416:21dominicmah, OK :)#2020-10-2416:10dharriganI would love to see deps.edn support some type of user home substitution. I have the a main $HOME/.clojure directory, in there I have my deps.edn file, that I pull in other libraries with that are all local root (i.e., setting up a REPL with some middleware etc..). Right now, since I’m on Linux I have to put an absolute path in, i.e., :base {:extra-deps {local-base/local-base {:local/root "/home/david/.clojure/libs/local.base" :deps/manifest :deps}}#2020-10-2416:10dharriganI cannot bring that across to my mac environment#2020-10-2417:06alexmillerMaking the .clojure/deps.edn portable across machines wrt paths is not a goal - imo if you’re on a different machine, you will need to adjust to that machine#2020-10-2417:07alexmillerWhich is not to say we won’t ever add something like this, but I’m not eager to do so#2020-10-2416:10dharrigansince that is /Users/home/david#2020-10-2416:11dharriganI thus have to maintain two separate deps.edn files that are near identical, apart from the path stuff.#2020-10-2416:12dharriganbut and I’m just trying this out…having it as a relative path may work!#2020-10-2416:13dharrigani.e., :base {:extra-deps {local-base/local-base {:local/root "libs/local.base" :deps/manifest :deps}}#2020-10-2416:14dharrigannope#2020-10-2416:14dharriganSwitching to a relative path, ends up with#2020-10-2416:14dharriganExecution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate local/rebel/main__init.class, local/rebel/main.clj or local/rebel/main.cljc on classpath.
#2020-10-2416:15dharriganwhich leaves me with having to put the absolute path in my deps.edn, and maintaining two separate deps.edn (which I suspect, if I was also on windows would incur more mangement)#2020-10-2416:18dharriganIt would be nice IMHO if deps supported at least having :local/root resolve the user’s home.#2020-10-2416:27dominicmHmm, I'm slightly surprised that resolution isn't relative when doing that. The personal tooling case is the only one I have so far too :)#2020-10-2416:57alexmillerI’m not, that’s what I’d expect - top level deps are relative to where you’re running clj#2020-10-2416:59dominicm@alexmiller I suppose they could be resolved with a relative *dir* like when doing transitive :local/root though?#2020-10-2416:59dominicm(obviously there's problems there due to the merging)#2020-10-2416:59alexmillerYeah, that’s problematic#2020-10-2417:10dominicmI suppose paths could be canonicalized pre-merging.#2020-10-2418:13borkdudethat's what I do in clj-kondo config. Each config can refer to local dirs relative to itself and these get resolved during config reading.#2020-10-2418:14borkdudethis way you can have hooks in your home dir config, not relative to your project#2020-10-2418:44alexmillerThat’s possible, I’ll think about it #2020-10-2418:59dharriganthat would be nice#2020-10-2418:59dharrigan🙂#2020-10-2608:12onetomi have an alias defined in my deps.edn as follows:
:ion-dev
{:extra-deps {com.datomic/ion-dev {:mvn/version "0.9.276"}}
:main-opts ["-m" "datomic.ion.dev"]}}
if i run /nix/store/6n60jkgpxb1a95wjx3ikhn9nvp5w33yh-clojure-1.10.1.727/bin/clojure -A:ion-dev -Sdeps '{:deps {nrepl {:mvn/version "0.8.0"}...' -m nrepl.cmdline ... it complains
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
WARNING: When invoking clojure.main, use -M
does this mean
1. i have to move the :main-opts into a different alias?
2. the command line should have a -M (without any aliases as a parameter) if i just want to run clojure.main?#2020-10-2608:16onetomim wondering what should be the naming convention for aliases in cases when one want to use some extra dependency both as just a library and as a command-line tool...
-A:ion-dev -M:ion-dev/run maybe?#2020-10-2608:40onetomam i supposed to combine aliases which doesn't contain :main-opts with aliases which does?
like -M:ion-dev:ion-dev/run?
when my deps.edn has:
:ion-dev
{:extra-deps {com.datomic/ion-dev {:mvn/version "0.9.276"}}}
:ion-dev/run
{:main-opts ["-m" "datomic.ion.dev"]}}#2020-10-2608:46dominicm@onetom needs a -M before the -m, so clojure -A:ion-dev -Sdeps … -M -m nrepl.cmdline …#2020-10-2610:29practicalli@onetom it is not required to have separate aliases, you can use the original :ion-dev alias and just use -M instead of -A . However, the -M (and -A flags) should come after the -Sdeps option and before the -m option (the -A flag does still work out of position though)
clojure -Sdeps '{:deps {nrepl {:mvn/version "0.8.0"}...' -M:ion-dev -m nrepl.cmdline
It is also possible to chain the aliases under -M , especially if there is an alias for nrepl too (this example taken from https://github.com/practicalli/clojure-deps-edn/)
:middleware/nrepl
{:extra-deps {nrepl/nrepl {:mvn/version "0.8.2"}}
:main-opts ["-m" "nrepl.cmdline"]}
And then simplify the command
clojure -M:ion-dev:middleware/nrepl
#2020-10-2612:02onetomsure, but i need the :extra-path in :ion-dev on its own too, without the :main-opts and so far i haven't found any option combination which wouldn't print some kind of warnings.#2020-10-2612:38onetomi went thru the new deps and cli docs and found this section:
https://clojure.org/reference/deps_and_cli#_replace_project_environment_tool
but it's unclear to me what does `tool` process refer to.#2020-10-2612:55practicalli:replace-deps will just use the deps specified in the alias, dropping any libraries from the main :deps section. This is useful if you just want to run an external tool, e.g. such as clj-new that creates a new Clojure project.
Using the -M option only calls the main namespace from the last alias in a chain, so with -M:ion-dev:middleware/nrepl only the main namespace from :middleware/nrep is called and the :ion-dev is simply adding the library as an additional dependency.
I didnt see any other main namespaces on the example shared.
I am not clear on why an :icon-dev with only extra paths is required in the scenario shared so far.#2020-10-2613:07onetomto start a development repl (with cursive or cider)#2020-10-2613:08onetombut i want to retain the option of running datomic.ion.dev/-main, since that's the official way to deploy datomic cloud projects.#2020-10-2613:09onetomwhen i have a repl running already (started with repl related :main-opts), im just using datomic.ion.dev/{push,deploy,deploy-status} directly#2020-10-2613:33alexmillerThere is no option to use the extra-deps but not the main-opts from an alias#2020-10-2613:33alexmillerYou should separate those into two separate aliases if you need that#2020-10-2614:09arohnerSomething in my deps is using a version range or SNAPSHOT because every time I run clj, it runs
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml
. Is there a clj command that will help me find which dep that is?#2020-10-2614:11arohnerit’s not -Strace or -Stree, because those only tell me the resolved version#2020-10-2614:16alexmillerif you do tree, it's probably whatever is above that dep in the tree ?#2020-10-2614:17alexmillerI don't think I have any other magic command to tell you more unfortunately#2020-10-2614:20arohnerthat helps some, thanks#2020-10-2614:21arohnerI was able to find the dep depending on cloudwatch (amazonica), but something else is causing
Downloading: org/clojure/clojure/maven-metadata.xml
. I guess I can wipe out ~/.m2/repository and do some grepping#2020-10-2614:22alexmilleralternately, use -Sdeps '{:mvn/local-repo "tmp"}'#2020-10-2614:22alexmillerbuild up a new tmp repository instead#2020-10-2615:58arohnerfound it. de.kotka/lazymap specifies:
<dependency>
<groupId>org.clojure</groupId>
<artifactId>clojure</artifactId>
<version>[1.2,1.6)</version>
<scope>compile</scope>
</dependency>#2020-10-2615:59arohnershould that always trigger a snapshot check?#2020-10-2616:04alexmillerall version ranges will trigger a resolution to the "newest" stable version#2020-10-2616:05arohnereven when bounded like that?#2020-10-2616:05alexmilleryes#2020-10-2616:06alexmilleryou could :exclude org.clojure/clojure down that path#2020-10-2616:06alexmilleralthough based on where the canonicalization happens, that may not help#2020-10-2616:09arohnerOn https://clojure.org/reference/deps_and_cli, I don’t see any definition for the contents of the :exclusion key#2020-10-2616:11alexmillerthat may be underdocumented :)#2020-10-2616:11alexmiller:exclusions [org.clojure/clojure]#2020-10-2616:11alexmillerin the coordinate#2020-10-2616:11alexmillerit looks like lazymap 3.1.1 dropped the version range#2020-10-2616:14alexmillerI don't think exclusions will help you here#2020-10-2616:15alexmillerat the point where expansion goes to resolve the deps, it's resolving all of them, the exclusions and stuff are handled at a higher level#2020-10-2616:15arohnerPutting lazymap 3.1.1 as an explicit dependency fixed it for me#2020-10-2616:15alexmilleryeah, that should work#2020-10-2616:15arohnerlazymap isn’t a dep of my project, it’s a dep of a dep. Putting lazymap in :override-deps doesn’t seem to do anything. Is it supposed to?#2020-10-2616:39seancorfield@arohner :override-deps only works inside an alias#2020-10-2617:02arohnerthanks#2020-10-2711:07mkvlrHey 👋 We’d like to bring in modules from another private repo using gitlibs. Ideally we’d want to use ssh auth. There’s no way to add support for another coordinate or get it to use tools.gitlibs from https://github.com/ghadishayban/tools.gitlibs/pull/1 without forking tools.deps (and having to distribute it to everybody on the team), is there? Or any other solutions to this problem that folks can recommend? Thanks!#2020-10-2711:11borkdudeIf all else fails, you could manually pull the code and add it via local/root?#2020-10-2711:14mkvlryep, guess that’s one option. Was also looking at your https://github.com/borkdude/deps.clj since we do require babashka on the project but the compute classpath part still happens on the jvm using the normal t.d.a jars right?#2020-10-2711:14borkdudecorrect#2020-10-2711:16borkdudeit pulls tools jars that are exactly the same as the jars that clj uses. however, we could make that configurable ;)#2020-10-2711:17borkdudebut maybe best to not create forks of official tools when it can be fixed in those official tools#2020-10-2711:18mkvlrright#2020-10-2714:04kirill.salykinhi
please advice - is there a way to include some java source files and use it from the clj?
for instance I want ot have this file used https://dzone.com/articles/java-tips-creating-a-monitoring-friendly-executors
is it possible to achieve with a c.t.d.?
thanks!#2020-10-2714:06alexmilleryou can compile the java source into java class files, then add the path containing those to your paths#2020-10-2714:07alexmilleryou can either use javac (the Java compiler) directly or there are various tools available to help with that https://github.com/clojure/tools.deps.alpha/wiki/Tools#2020-10-2714:10kirill.salykini see, thanks for the options#2020-10-2714:29borkdudewould it be an idea to extend -X with being able to pass a single map?
clj -X:graph graph :deps '"mydeps.edn"' :trace true :output '"trace"'
clj -X:graph graph '{:deps "mydeps.edn" :trace true :output "trace"}'
The quoting of strings seems a little clumsy on the command line, so quoting the entire map can deal with that as a whole.#2020-10-2714:30borkdudeI've been adopting the latter style for more tools where I just pass the entire map as --opts, e.g.:
$ echo '[1 2 3]' | puget --opts '{:print-color false}'#2020-10-2714:31borkdudeAnother question: is the -X CLI parsing logic available as a library? I think this could be used in other places than clj as well (e.g. GraalVM compiled binaries).#2020-10-2714:38alexmilleranswering the last question first, no but that's something we're actively thinking about - I think it's something that could end up in Clojure itself (like clojure.main) but I am still actively changing the interface to it and will need further work before I think it's there#2020-10-2714:40alexmilleron the former question, that's a good question, will think about it#2020-10-2714:41alexmillerI mean really the idea was to put the base map in alias data and then just overrides on the cli, but some tools would benefit more than others from this#2020-10-2721:13dominicmI keep thinking I've missed a way to have some kind of :deps-aliases in an alias to use the extra deps from another alias, similarly to how I can with :extra-paths.#2020-10-2721:16alexmillerI'm confused by what you mean about :extra-paths#2020-10-2721:17alexmillerunless you mean paths as alias data#2020-10-2721:17dominicmI mean that I'm surprised there's no equivalent to :extra-paths [:build-paths] such as :extra-deps [:build-deps]. Obviously that syntax doesn't work, but that general idea.#2020-10-2721:17dominicmI do yeah :)#2020-10-2721:17alexmillerah, ok#2020-10-2721:18alexmillerwould certainly complicate the (already too complicated) merging logic#2020-10-2721:19dominicm:extra-deps [:build-deps {…}] maybe…#2020-10-2721:21alexmillerI'd probably be tempted to define something mappy instead so the merging would continue to work (like destructuring syntax or spec selects)#2020-10-2721:22dominicm:extra-deps {:clojure.tools.deps/aliases [:build-deps] my.cool/lib {}} ?#2020-10-2721:22alexmillerwell, I'm not going to design it right now :)#2020-10-2721:22alexmillerif you wanted to file a jira or an ask request, would be glad to keep that idea around#2020-10-2721:25dominicmI'll open a JIRA. :)#2020-10-2721:26alexmillerthx#2020-10-2915:41dpsuttonworking to give an error message to verify that the tools deps version is too old to support -M -m. Does anyone know offhand if a version supported -X but not -M -m? Hoping can check something real quick with -X and that means it necessarily supports -M#2020-10-2915:44dpsuttonas it stands, added a simple (ns check) (defn -main [& _] (System/exit 0)) but would prefer to not need a new main and new ns if X and M go hand in hand#2020-10-2916:27dpsuttonie, could just do clojure -X clojure.core/identity :a :a and if that returns 0 then good to go for -M -m our.stuff#2020-10-2916:38lread@dpsutton I expect this is not what you are looking for, but I write my scripts in Babashka where I find a Clojure version check to be explicit and straightforward: https://github.com/lread/rewrite-cljc-playground/blob/60ad8c118a86221e35692f2ce10281e7cef0f8a4/script/helper/env.clj#L39#2020-10-2916:39practicalli@dpsutton https://clojure.org/releases/tools might have the info about -X but not -M -m support... although I believe -X support wasnt added until https://clojure.org/releases/tools#v1.10.1.697 and -M -m was in by then.
I use https://clojure.org/releases/tools#v1.10.1.697 as the minimum requirement for Clojure CLI tools with my aliases and content from Practicalli#2020-10-2916:42seancorfield-M has been in the CLI for a long time but it's meaning changed around 1.10.1.697#2020-10-2916:43seancorfieldThe difference between 1.10.1.561 (stable) and the later version is just that -M now respects resolve-args and classpath-args in aliases as well as main-opts. Previously, it ignored resolve-args and classpath-args. Does that help @dpsutton?#2020-10-2916:45dpsuttonYeah that does. I think that means we should use the check for X support to make sure we have a recent enough version#2020-10-2916:45dpsuttonAlthough I think our use case is simple enough that as long as M support is there it’s fine#2020-10-2916:59seancorfield@dpsutton I just ran some tests on a bunch of prerelease versions between the last two stable releases, and there are several where that command returns a zero status but doesn't actually work.#2020-10-2917:00seancorfield(but I believe they would support -M with aliases by that point... I just haven't tested that 🙂 )#2020-10-2917:02dpsuttonIn what way do they not work?#2020-10-2917:02dpsuttonAnd thanks so much!#2020-10-2917:02seancorfield(! 1626)-> /usr/local/Cellar/clojure\@1.10.1.636/1.10.1.636/bin/clojure -X clojure.core/identity :a :a
No alias specified with -X
Thu Oct 29 09:58:15
(sean)-(jobs:0)-(/Developer/workspace/wsmain/build)
(! 1627)-> echo $?
0
#2020-10-2917:03seancorfieldThe ability to pass the fn to execute on the command-line came fairly late in that set of prerelease builds.#2020-10-2917:04seancorfield1.10.1.672 seems to be the first prerelease build that allowed it.#2020-10-2917:05dpsuttonOh ok. It’s only prereleases? If so they can diagnose it on their own :)#2020-10-2917:05dpsuttonBleeding edge is sometimes bleeding#2020-10-2917:06seancorfieldYeah, 1.10.1.561 (previous stable) throws an exception and gives a non-zero exit status.#2020-10-2917:06seancorfield(so does 1.10.1.590 which was the first of the prerelease builds)#2020-10-2917:06dpsuttonBeautiful. How do you change versions so quickly? Would have done that if I knew a good way to swap them around easily#2020-10-2917:07seancorfieldI installed them by version with brew and then I can use the full path (as shown above) to run any version I want 🙂#2020-10-2917:08kirill.salykinhow are you doing this?#2020-10-2917:09kirill.salykinjust brew install @version ?#2020-10-2917:09seancorfieldbrew install clojure/tools/ per the CLI install docs#2020-10-2917:09kirill.salykinclear, thanks a lot!#2020-10-2917:10seancorfieldSee https://github.com/clojure/homebrew-tools (which is linked from the CLI install docs on the CLI guides page)#2020-10-2917:07seancorfield{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "fe9d9291948b8c9bbecfd0cfced0cfd0cbc8cf"}, :content ("[email protected]")}#2020-10-2917:08seancorfieldI didn't have 561 until just now, but installed it by version to test.#2020-10-2917:14dpsuttonOh I didn’t know it left older versions laying around#2020-10-3021:04borkdudeclojure -A:foo and clojure -M:foo without main-opts do the same thing: invoke a REPL with alias foo merged in -- right? Should -M:foo give a warning when you forget to supply main-opts in deps.edn maybe?#2020-10-3021:05borkdudeor should -A not invoke a REPL since that's a clojure main invocation?#2020-10-3021:15seancorfield-A is for REPL invocation, according to the docs.#2020-10-3021:16borkdudeso -A and -M overlap there I guess#2020-10-3021:17seancorfieldI suspect that's a bit of a grey area given that :main-opts ["-r"] works to run a REPL, yes?#2020-10-3021:17borkdudewell, I invoked an alias with -M while I forgot to provide :main-opts in deps.edn, so it invoked a REPL for me#2020-10-3021:17borkdudethat's what prompted these maybe too far fetched thoughts#2020-10-3021:18seancorfieldYeah, I would sort of expect -M to at least give a warning if no :main-opts are provided. @alexmiller?#2020-10-3021:19seancorfield(! 651)-> fgrep :repl deps.edn
:repl {:main-opts ["-r"]}
(! 652)-> clj -M:repl
Clojure 1.10.1
user=> ^D
(! 653)-> clj -A:repl
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
Clojure 1.10.1
user=> ^D#2020-10-3021:20seancorfield(so, yeah, you can specify -r explicitly as a :main-opts item)#2020-10-3021:21seancorfieldAnd the corollary:
(! 656)-> fgrep :dummy deps.edn
:dummy {}
(! 657)-> clj -M:dummy
Clojure 1.10.1
user=> ^D
(! 658)-> clj -A:dummy
Clojure 1.10.1
user=> ^D#2020-10-3021:22alexmilleryou can supply main-opts on the command line, so -M does necessarily have to have :main-opts#2020-10-3021:22alexmillerclj -M:my-alias-with-just-deps -m my.ns#2020-10-3021:23seancorfieldYes, but that's not the case we're discussing here.#2020-10-3021:23alexmillerwell, what is?#2020-10-3021:23seancorfieldSee the two console examples I pasted. No command-line main-opts there.#2020-10-3021:23alexmiller-M invokes clojure.main. clojure.main does not require opts.#2020-10-3021:25seancorfieldFair enough. The "implicit" -r option 🙂#2020-10-3021:26seancorfieldWhy draw the distinction in the docs then that -A is for REPL execution?#2020-10-3021:26borkdudebecause -M might not invoke a REPL when main-opts are given#2020-10-3021:26alexmillerreally, it can be used to supply aliases for anything but it's only useful with REPL (because -M can be used for main and -X can be used for functions)#2020-10-3021:27alexmiller(and for allowing legacy stuff to continue working)#2020-10-3021:27seancorfieldI guess this will become less confusing once -A stops executing :main-opts#2020-10-3021:27alexmilleryes#2020-10-3021:28seancorfieldI'm curious how you plan to distinguish between -A for a REPL and -M executing clojure.main which might pick up other main opts?#2020-10-3021:30alexmillerwhat do I need to distinguish?#2020-10-3021:31seancorfieldI mean: how is -A going to start a REPL without running clojure.main?#2020-10-3021:31alexmillerit already does this#2020-10-3021:31alexmillerit invokes clojure.main#2020-10-3021:31alexmillerthat's an implementation detail#2020-10-3021:32alexmillerclj without -M or -X will start a repl, you let me figure out how to do that :)#2020-10-3021:33seancorfieldSo, in the absence of :main-opts in an alias, both -A and -M will work identically, always? i.e., they will both run clojure.main and they will both start a REPL?#2020-10-3021:28borkdudeI would maybe prefer -M to trigger a warning when there is no :main-opts [] when you want to explicitly pass 0 arguments to main (i.e. invoke a REPL)#2020-10-3021:29seancorfield@borkdude I don't think that is going in the right direction...#2020-10-3021:30seancorfieldIt seems that the only way to avoid continued confusion is to have clojure.main without arguments not start a REPL -- and require -r for that? 🙂#2020-10-3021:30alexmillerI don't follow#2020-10-3021:31borkdudenever mind#2020-10-3021:33alexmilleryes?#2020-10-3021:34seancorfieldI guess you could always have a new entry point for a REPL and have -A invoke that...?#2020-10-3021:35alexmillerI don't understand what problem is bothering you#2020-10-3021:35seancorfield(or rather "have the absence of -M and -X invoke that")#2020-10-3021:35alexmillerthe absence of -M or -X means: invoke a repl#2020-10-3021:36alexmillerin other words, the default behavior of clj is: invoke a repl#2020-10-3021:36seancorfield(! 659)-> clj -A:dummy -r
WARNING: When invoking clojure.main, use -M
Clojure 1.10.1
user=> ^D
#2020-10-3021:36alexmillerok, you're outside the bounds of documented behavior#2020-10-3021:37alexmillerthat's legacy support#2020-10-3021:37seancorfield-r is a documented main-opt that causes clojure.main to start a REPL, isn't it?#2020-10-3021:37alexmilleryes, but when we stop accepting main opts outside of -M, that's no longer a thing#2020-10-3021:38alexmillerthere may be some other set of accepted arguments in the repl case that have nothing to do with clojure.main#2020-10-3021:38seancorfieldOK, so there will need to be a change in the behavior of clojure.main at some point to "fix" this?#2020-10-3021:38alexmillerno#2020-10-3021:38seancorfieldclojure.main's help says
-r, --repl Run a repl
#2020-10-3021:38alexmiller"clj starting a repl" is distinct from clojure.main#2020-10-3021:39alexmilleryes, the mechanism clj currently uses to start a repl is clojure.main#2020-10-3021:39alexmillerbut that's an implementation detail#2020-10-3021:39seancorfieldSo there will be a non-`clojure.main` entry point that clj can use to start a REPL?#2020-10-3021:39seancorfield(otherwise this confusion is going to continue)#2020-10-3021:40alexmillerwhat I'm saying is - I am promising clj will start a repl. the end.#2020-10-3021:40alexmillerand I'm promising that clj -M will run clojure.main#2020-10-3021:40alexmillerand that clj -X will invoke a function with a map#2020-10-3021:41seancorfieldbut you're not promising that clojure.main will not support -r at some future point :rolling_on_the_floor_laughing:#2020-10-3021:41alexmillerthere are no plans to make clojure.main do anything different than what it does now#2020-10-3021:42alexmillerin the before times, clj always ran clojure.main. that is no longer the model#2020-10-3021:43alexmillerthe new model is:
• by default, run a repl
• with -M, run clojure.main
• with -X, execute a function with a map#2020-10-3021:43alexmillerand "run a repl" does not say anything about clojure.main#2020-10-3021:45alexmillerif you want to explicitly run clojure.main to run a repl, then clj -M or clj -M -r will do that#2020-10-3021:45seancorfieldThe confusion arises because the docs talk about "using -A (for REPL invocation)" when it still runs :main-opts!#2020-10-3021:46alexmillerwell, we're in a bridge period between the two models#2020-10-3021:46seancorfieldThe sooner -A stops running :main-opts the better...#2020-10-3021:46seancorfield(mind you, even in that future world, I think @borkdude’s confusion will still be present, yes?)#2020-10-3021:47alexmillerif the confusion is that you can use clojure.main to run a repl, then yes#2020-10-3021:48alexmillerbut it's easy to reason about if we are properly conveying that -M == run clojure.main#2020-10-3021:48seancorfieldArgh, I'm being bitten by caching here 😞#2020-10-3021:49seancorfield(! 665)-> clj -Sforce -A
Missing required argument for "-A ALIASES"
(! 666)-> clj -Sforce -A -r
Missing required argument for "-A ALIASES"
#2020-10-3021:49seancorfield^ That makes more sense.#2020-10-3021:49seancorfield-M and -X can be invoked without an alias, -A cannot.#2020-10-3021:49alexmillerwhat does that have to do with caching? -A always requires an alias#2020-10-3021:50seancorfieldBecause this happened before I added -Sforce:
(! 662)-> clj -A
Clojure 1.10.1
user=>
(! 663)-> clj -A -r
WARNING: When invoking clojure.main, use -M
Clojure 1.10.1
user=> ^D
#2020-10-3021:50seancorfield(note the command history numbers there!)#2020-10-3021:51alexmilleroh, well that's just a bug then#2020-10-3021:54seancorfieldHere's the repro for that bug:
(! 674)-> clj -Sforce -A
Missing required argument for "-A ALIASES"
(! 675)-> clj -A
Clojure 1.10.1
user=>
#2020-10-3021:55alexmillerfixed#2020-10-3021:56borkdudeSo the point earlier was: invoking -M with an alias that doesn't have :main-opts assumes that :main-opts is implicitly []. My question was: should this be less implicit, to avoid people ending up in a REPL when invoking -M with an alias that wasn't intended for -M at all#2020-10-3021:57alexmillerno, because :main-opts is not required#2020-10-3021:57borkdudeok#2020-10-3021:58seancorfield@borkdude in the following :test has no :main-opts -- which of these would you expect to get warnings on and why?
(! 676)-> clj -Sforce -M:test -r
Clojure 1.10.1
user=> ^D
(! 677)-> clj -Sforce -A:test -r
WARNING: When invoking clojure.main, use -M
Clojure 1.10.1
user=> ^D
(! 678)-> clj -Sforce -M:test
Clojure 1.10.1
user=> ^D
(! 679)-> clj -Sforce -A:test
Clojure 1.10.1
user=> ^D
#2020-10-3021:58alexmillerwe could do something like - if you supplied -M but did not supply either :main-opts or main-opts on the command line, then either warn or error at that point#2020-10-3021:58alexmillerbut honestly that's difficult to check right now due to where that knowledge of different parts of that is#2020-10-3021:58borkdudeyes, that's exactly the case I was referring to#2020-10-3021:58seancorfieldSo -M wouldn't silently start a REPL unless you said -M -r ? I think that would be reasonable.#2020-10-3021:59seancorfield(if it could be checked)#2020-10-3021:59alexmillerand then -M means run clojure.main except for this one case where it fails#2020-10-3021:59borkdudeyeah, I guess it's not worth changing#2020-10-3021:59seancorfield(I'd be against an error there but I think a warning to remind folks might be worthwhile)#2020-10-3022:00borkdudemain opts can be supplied on the command line, so not providing arguments means the same as :main-opts [], but you can also do that with -A which is the preferred way of invoking a REPL maybe#2020-10-3022:01alexmilleryes#2020-10-3022:01alexmiller@seancorfield 1.10.1.735 if you want to check the -A thing#2020-10-3022:01seancorfieldBut I suspect there's no point trying to second-guess what folks will find confusing until after we have a CLI version where -A no longer uses :main-opts#2020-10-3022:01seancorfield@alexmiller Thanks for the swift fix!#2020-10-3022:03seancorfield(! 680)-> clj -Sforce -A
-A requires an alias
(! 681)-> clj -A
-A requires an alias
confirmed fixed!#2020-10-3022:04alexmillerI did not take the time to try on Windows, but it might already catch this case due to the different parsing. I'll check that before we get to a stable release#2020-10-3022:04alexmillerI'm shutting down, have a good Halloween weekend y'all#2020-11-0117:54holyjakHello! I am sorry, I obviously am not good enough in reading docs. What do I need to do to be able to run my project with just clj -X without arguments? I have tried
{:deps {...}
:exec-fn clj-tumblr-summarizer.main/-main}
but that complains "No function found on command line or in :exec-fn"
(https://clojure.org/guides/deps_and_cli#_using_a_main does not mention how to hardcode the function. I have also tried https://clojure.org/reference/deps_and_cli#_running_a_main_or_script and add :main-opts ["-m" "clj-tumblr-summarizer.main/-main"])#2020-11-0118:04practicalliYou need to define an alias to use with the -X flag.
My understanding is the -X flag should be used with an alias, as I havent seen any examples of using :exec-fn as a top level key. From the error you are getting, this seems to be a correct...
There are plenty of examples of :exec-fn , :exec-args and :ns-default in the practicalli/clojure-deps-edn
https://github.com/practicalli/clojure-deps-edn#2020-11-0118:05vlaaadI think exec-fn has to be in alias, and -X requires alias#2020-11-0118:06practicallisnap 🙂#2020-11-0118:07practicalliYou can specify :exec-fn on the command line I believe, if you really dont want an alias
Edit: hmm, I am not sure that is right now. You can specify a function, but still has to be with an alias.
Edit: as borkdues says below, if its a fully qualified function on the class path, then the command line is enough - need to add some good examples to the book so I dont forget#2020-11-0119:00borkdudeThat's not true though:
$ clj -X clojure.core/prn :foo 1 :bar 2
{:foo 1, :bar 2}#2020-11-0119:28seancorfieldYou can't run just clj -X on its own. You either have to provide an alias or you have to specify the function to call on the command-line.#2020-11-0119:28seancorfieldJust like :main-opts can't be at the top level in deps.edn, :exec-fn, :exec-args, :ns-default, etc all need to be under an alias.#2020-11-0119:31seancorfieldThere are only a few keys valid at the top level @U0522TWDA
:deps - map of lib (symbol) to coordinate
:paths - vector of paths
:aliases - map of alias name to alias data
provider-specific keys for configuring dependency sources
(that last one is stuff like :mvn/repos)#2020-11-0119:32seancorfieldSee https://clojure.org/reference/deps_and_cli#_operation (just after the link you already posted).#2020-11-0119:39holyjakThanks a lot!#2020-11-0204:27olivergeorgeNot sure if this is important/interesting but I hit what seems to be a gitmodules related problem. https://gist.github.com/olivergeorge/abc428669538622b88dd13cd876aceae happened with this deps.edn file:
{:deps {clj-statecharts {:sha "942eb942594cdd6876556cf212d59cff245fa9fe" :git/url ""}}}#2020-11-0214:27alexmillerwe have a ticket for git submodules actually https://clojure.atlassian.net/browse/TDEPS-126#2020-11-0214:27alexmillerhave been through a few rounds on the patch, but I haven't looked at it recently#2020-11-0222:23olivergeorgeThanks Alex#2020-11-0204:27olivergeorge#2020-11-0211:32borkdudeGot part 1 working of clj-kondo deps.edn linting:#2020-11-0222:21alexmillerwhen you look at a deps tree (`clj -Stree` , lein deps :tree , mvn dependency:tree ), what question are you trying to answer?
1. what are the set of deps I'm using?
2. what version of a particular dep am I using?
3. what are the transitive deps of a particular library?
4. is a particular dep version out of date?
5. is some dep I'm expecting not on my classpath?
6. why is some dep on my classpath?#2020-11-0222:33andy.fingerhutYou should give higher weight to the answers of others than me (I'm not using Clojure in production myself), but usually 1 3 6 and sometimes 2.#2020-11-0222:34andy.fingerhutSeems like a question you could reasonably put on State of Clojure Survey 2021 if you were so inclined 🙂#2020-11-0222:49alexmillerso, interestingly you can't answer #3 or to some extent #6 correctly based on the result of any of those tools#2020-11-0222:51alexmillerin many cases deps show up at multiple places in the "tree" (really, a graph) and all print it only at one location (trivial example, clojure is usually included N times but only shows up once)#2020-11-0223:14nateI don't know if this falls into 5 or 6, but I most often: what is forcing dep x to be version y?#2020-11-0223:15natesometimes I use it for 1, but more often that's -Spath with a grep to find a particular dep#2020-11-0223:37dorabMostly 6 and 2 for me. 5 and 3 sometimes.#2020-11-0307:25dharrigan6 3 5 1 and lesser 4 2 since I use antq regularly to ensure my deps are up-to-date.#2020-11-0308:02vlaaadI guess that might be a variation of 6, but “what deps depend on this dep?”#2020-11-0308:40dominicm1 2 5 6. With the additional use case of chasing "which dependencies transitively depend on A, and need to have exclusions added." That's particularly annoying in clj as it only shows the dependency that was retained.#2020-11-0309:14rickmoynihanI’d say all of them to some extent, but mainly 5, 6 and 2…
1 is more often asked as a general high-level question; so I’d normally just look at the deps.edn to get a rough overview, and overview of the intention.
I usually use -Stree etc to help answer a slightly different question, which is “What deps are being chosen and why”. This usually being a combination of the above options. I think the distinction is often when I’m running -Stree it’s as a debugging tool, and I don’t know what the problem is yet (i.e. it could be 1 because I’ve started with the wrong aliases, it could be 2 because the wrong version of a transitive dep is being selected, it could be 3 if we’re accidentally explicitly depending on a transitive dep which has been removed or isn’t available in the alias combination. Similarly for 5 sometimes 6 if an unused dep is causing a transitive dep to be a different version).
For example all of those questions might uncover the reason behind a ClassNotFoundException, or perhaps a method being called with the wrong arguments etc. But until you run -Stree you don’t know which it’s going to be.#2020-11-0309:17rickmoynihanAhh just seen @U04V70XH6 said essentially the same thing in the main channel#2020-11-0222:24alexmillersome possible answers there, but that's an open set#2020-11-0222:52dpsuttonwanting other use cases or to chime in with which of the above are most useful?#2020-11-0222:53alexmillerjust what it says above - when you reach for this, what question are you usually trying to answer?#2020-11-0222:54alexmilleror separately, what are some dep tree related questions you would like to be able to answer?#2020-11-0222:55dpsuttonused it today to diagnose a bad release of piggieback. it had some extra junk tucked in the jar and the cleanest repo was to clj -Sdeps '{:deps {cider/piggieback {:mvn/version "0.5.1"}}}' -Stree. then rebuilt the jar locally and checked with clj -Sdeps '{:deps {cider/piggieback {:local/root "target/piggieback-0.5.1.jar"}}}' -Stree that the resultant jar was clean so it was just a deploy problem#2020-11-0222:56alexmillerso this is really #1 in a diff scenario#2020-11-0222:56dpsuttonyes. 1 and 3#2020-11-0222:58alexmillerwell it doesn't really answer #3 - that's often a lie as it's thinning a graph into a tree#2020-11-0222:55ghadiUsually 1 & 3 for me#2020-11-0223:01seancorfieldMy two scenarios are: 1. trying to debug a problem that turns out to either be an unexpected transitive dependency or an unexpected version of one of them (which is a combination of several of those six things to one degree or another) and 2. as part of my script that attempts to determine #4 above (by synthesizing a new project from the output of clojure -Stree and then running clojure -Stree on that and seeing what versions changed -- but only because none of the "outdated deps" tools out there work properly in our monorepo setup at work).#2020-11-0223:03seancorfieldI'd say scenario 1 is a mix of #3, #5, and #6 primarily (and I realize that the tree isn't a full/accurate representation of the actual graph, but it's "close enough" and can always be used selectively with -Sdeps to select specific top-level deps to examine further).#2020-11-0418:25rickmoynihanSo I figure this might be of some interest to people on this channel.
I’ve forked @slipset’s deps-deploy to support a number of new features and have created PRs for each of them:
1. Configuration via :exec-args and all that means (i.e. overriding parameters via the CLI) https://github.com/slipset/deps-deploy/pull/18
2. Support for private S3 buckets via s3-wagon https://github.com/slipset/deps-deploy/pull/19
3. A new feature to allow overriding the version tag in the pom.xml with whatever you supply as a :mvn/version in :exec-args or more commonly through a command line override. My hope is this should help generating new artifacts with new versions in CI. https://github.com/slipset/deps-deploy/pull/20
I figured I’d mention this in case anyone has any comments or feedback here.
It’s yet to be documented, and slipset is yet to agree to any of it, and there are some other things I’d like to add too like the ability to influence more of the parameters that are passed into pomegranate/aether (which would allow for example configuration of proxies etc in deps.edn), but I thought it might be worth mentioning.#2020-11-0419:17dominicmThe other common override along with version is the scm tag (which is usually v${version}).#2020-11-0421:38rickmoynihanWhat are the usecases for overriding it?#2020-11-0421:56dominicmThe tag points at the commit or git tag with the release pertains to. So you should override it when releasing so, e.g. Cljdoc will pick up the write commit to read docs from.#2020-11-0422:00rickmoynihanthanks makes sense#2020-11-0422:05dominicmTotally going to move away from garamond to your thing.#2020-11-0517:34rickmoynihanI wonder if the override feature might be better placed in depstar?!#2020-11-0517:57dominicmI think there's a lot of ambiguity in pom processing right now#2020-11-0609:17rickmoynihanyeah that’s precisely my point…
Personally I feel like deps-deploy and depstar should probably be integrated into the same project. And use each others :exec-args keys etc..
Though I guess that depends on whether there are any plans for the cli tools to ever support task composition. (I know it’s been raised here a few times). Here’s hoping tools.build lets you compose tasks intelligently :crossed_fingers:
The project I’m working on wants to build and deploy multiple builds (essentially themed versions of the same thing), and to do that right now I need to fire up two JVMs for each build — and we might need 10 builds at some point… So that’ll be a lot of time wasted in CI starting/stopping JVMs. Not the end of the world; but might add a minute to the CI time; when it could be more like 10 seconds.#2020-11-0611:45dominicmSilly question, why not write a short clj file which calls them with the right args?#2020-11-0611:45dominicmAssuming a functional api#2020-11-0613:58slipsetJust found this https://github.com/applied-science/deps-library#2020-11-0614:41rickmoynihan> Silly question, why not write a short clj file which calls them with the right args?
Well this is my point. I can definitely do that, and likely will. But I’m only solving it for myself in one project. If you build a jar you almost always want to deploy it.
depstar and deps-deploy would ideally be aligned and integrated around common keys etc.
It looks like deps-library is essentially what that might look like, though with a traditional cli interface not an :exec-fn one.#2020-11-0614:42rickmoynihanThe iteration etc is fairly unique to my project; but the integration of deps-deploy and something like depstar is common enough that it can/should be combined in my mind.
If we had that it would make the job of my iteration/script even smaller (but also I think be widely useful)#2020-11-0616:32rickmoynihanWell it turns out this is a bit of a problem to doing it as a clj script:
https://github.com/seancorfield/depstar/blob/5b1f9f02b34d07f88b42cbf2c2109d04b829cd8b/src/hf/depstar/uberjar.clj#L525#2020-11-0616:34slipsetJust to be clear, I’d be happy to move deps-deploy in (almost) any direction that might help solve problems.#2020-11-0616:36rickmoynihanThat’s pretty much my thoughts on what should happen too…
I feel we need some community consensus on what that direction might be though.#2020-11-0418:25rickmoynihanAnyway that’s me done for the day… but will check back later / tomorrow.#2020-11-0421:01Eamonn SullivanHi all. I'm trying to use a library in my deps.edn that's on our corporate artifactory. This site uses ssl client certificates for authentication. In Scala/sbt (our normal code), we set java options such as -.ssl.keyStore=... -.ssl.keyStorePassword=... in the environment ($SBT_OPTS). But I can't seem to get this to work for clojure. I tried just directly passing these in like -.ssl.keyStore=... but it doesn't seem to work. I get a handshake_failure. Does this sound familiar to anyone? I can build the library I want locally and install it directly in ~/.m2, but that's an awkward way to work.#2020-11-0421:05ghadithe reason that doesn't work is because -J options apply to the application JVM that gets launched -- your program.
the dependencies themselves are fetched in a previous invocation that doesn't receive those -J options#2020-11-0421:06ghadiwill defer to @alexmiller on how to use custom TLS on the deps fetching#2020-11-0421:10Eamonn SullivanThanks, yeah, that makes sense. I can continue working (installing locally) for now -- just trying to make the workflow a bit less convoluted.#2020-11-0520:11borkdudeI merged the first batch of checks for deps.edn in clj-kondo to master.
Feel free to read through the tests to see what it does:
https://github.com/borkdude/clj-kondo/blob/a9df3965716025e93764d36100314edca30e5cee/test/clj_kondo/deps_edn_test.clj#2020-11-0616:33rickmoynihan@seancorfield: would you take a PR to refactor the bulk of this function into a new function that does everything except call System/exit?#2020-11-0616:33rickmoynihanhttps://github.com/seancorfield/depstar/blob/5b1f9f02b34d07f88b42cbf2c2109d04b829cd8b/src/hf/depstar/uberjar.clj#L525#2020-11-0616:34rickmoynihanoh actually looks like it’s not system-exit that is the problem… but shutdown-agents#2020-11-0616:35rickmoynihanwell the system/exit is also bad as it kills my repl when I make a mistake in the config#2020-11-0616:36seancorfieldIt calls System/exit in multiple places.#2020-11-0616:36rickmoynihanyeah so I’ve seen#2020-11-0616:36alexmillerI often try to separate the shutdown-agents/System.exit type stuff into a wrapper#2020-11-0616:36rickmoynihanyeah me too#2020-11-0616:36rickmoynihanthat’s what I’m suggesting#2020-11-0616:37seancorfieldYeah, I would normally but I haven't quite gotten there yet.#2020-11-0616:40seancorfieldThere's also the call to help-and-die in there. I'll give it some thought this weekend.#2020-11-0616:41rickmoynihanshall I make an issue for it?#2020-11-0618:17seancorfieldhttps://github.com/seancorfield/depstar/issues/45#2020-11-0713:32rickmoynihanThanks sean, sorry I didn’t get around to this yesterday.#2020-11-0718:44seancorfieldNP.#2020-11-0720:53seancorfield@rickmoynihan release 1.1.133 has this refactoring so you can use hf.depstar.uberjar/run* from the REPL now. Same args as hf.depstar.uberjar/run but it returns a hash map for success/failure. The old entry points all still call (shutdown-agents) or (System/exit 1).#2020-11-0720:53seancorfieldFollow-up in #depstar if you want.#2020-11-0616:41seancorfieldSure. It'll be a good reminder for me.#2020-11-0616:42alexmilleranother option is to parameterize the "on-exit" function#2020-11-0616:45borkdudeI usually write a -main and main. Main only returns the exit code (or a map with exit code) and -main does System/exit + shutdown-agents#2020-11-0616:45borkdudeI then use main in my tests#2020-11-0616:46rickmoynihanyeah that’s the same pattern I do… though this isn’t main but an :exec-fn not that it makes much difference.#2020-11-0616:47rickmoynihanit lets you essentially test command line opts etc#2020-11-0616:49borkdudeyes, this is how I test all my CLIs using the exact same input as from the command line#2020-11-0616:51rickmoynihanWhilst I’m hear, as you may have seen I’ve been tweaking deps-deploy to support the new -X stuff.
If you’re going to use deps-deploy however; you essentially need to use depstar (or similar), so it’d be good if these tools could be aligned a bit to respect the same keyword vocabulary.
How do people feel about that?
Similarly in the deps-deploy changes I made I thought it would be nice to use :mvn/version as it means the same as in tools.deps. However I also wanted :mvn/artifact-id and :mvn/group-id etc. Would those terms in principle be the same as tools.deps / tools.build etc might use?
Is it legitimate for me to coin keys in the mvn namespace?#2020-11-0616:52alexmillerto use in what context?#2020-11-0616:53alexmillerlike in calling with -X or something?#2020-11-0616:53rickmoynihanyeah precisely, my patches to deps-deploy allow you to override the :mvn/version via :exec-args and the cli, but also :mvn/artifact-id etc…#2020-11-0616:53alexmillerprobably won't line up to tools.build but I don't know if you should care#2020-11-0616:54rickmoynihanwell I don’t know what tools.build does… so hard to know if I should care or not 😀#2020-11-0617:03rickmoynihanI guess I have a slightly deeper question here though…
Which is essentially how deep does RDF’s influence over best practice around keywords in clojure go?
Rich has mentioned RDF publicly multiple times over the years. And clojure’s namespaced keywords etc are essentially equivalent to predicates etc. The promoted strategy to handling them has been “if they have the same meaning” it’s fine to reuse them (open world, open for extension etc…). However the linked-data world also has a convention of only coining new terms in namespaces you own. However clojure best practice glosses over that here. Does tools.deps own :mvn, is it legitimate for me to coin keywords in that namespace?#2020-11-0617:03rickmoynihanAlso is the concept of keyword vocabularies perhaps a useful one?#2020-11-0617:03dpsuttonWhat is "RDF"?#2020-11-0617:05rickmoynihanhttps://www.w3.org/TR/rdf11-primer/#2020-11-0617:08rickmoynihanTLDR; A web-scale datomic-like data model described as a set of W3C standards and practices for how people can build databases to support a global graph of knowledge#2020-11-0617:11rickmoynihan(Also acknowledged by Rich to be a big influence on the design of clojure and datomic too)#2020-11-0617:08alexmillerI do not consider tools.deps to "own" :mvn#2020-11-0617:09alexmillerqualifiers and their usage are contextual. spec argues for sufficient qualification to create a universe of unambiguous attribute definitions#2020-11-0622:46dominicmFwiw, I think this would be a valuable article on http://clojure.org. Explaining the purposes of namespacing and when you can skip it, when to use #db/id instead of #com.datomic/id Or #datomic/id.#2020-11-0617:10alexmillerin more limited contexts (inside your project or inside another appropriately scoped data context), less qualification may be sufficient#2020-11-0617:10alexmillerdeps.edn has a competing design constraint of wanting to be succinct as this is a format people are frequently writing#2020-11-0617:11alexmillerwe ended up leaning more on the latter constraint in that balance#2020-11-0617:13rickmoynihanThanks for that @alexmiller, that’s essentially my perception too.
What then do you think of collaborating around common shared definitions like :mvn/version etc and agreeing in principle on other ones to avoid future conflict in areas such as this?#2020-11-0617:14alexmillerfor the purposes of invoking tools, I don't think there's any need to collaborate#2020-11-0617:14alexmillerand I don't see it as conflict#2020-11-0617:16rickmoynihanwell it’s not a conflict if I chose :mvn/group-id and you choose :mvn/groupid, but it is irritating if we mean the same thing and our terms flow through the same channels.
It would be a potential conflict if we both independently chose :mvn/repository but I defined it as a map, and you a string; and they happened to flow through the same context.#2020-11-0617:18alexmillerI don't see them flow through the same channels or exist in the same context, which is why I say this is not a conflict#2020-11-0617:19alexmillerreally, I'd say neither of these tools "own" :mvn. use a qualifier you own if you want to own it.#2020-11-0617:19alexmillerotherwise, don't complain when it conflicts :)#2020-11-0617:26rickmoynihanWell that’s certainly pragmatic. I’ll take that as permission to follow your example and use :mvn/group-id here. I mean it seems sensible to follow similar conventions for human reasons.#2020-11-0617:34alexmillerI have no opposition to using that name :)#2020-11-0617:25seancorfielddepstar's arguments to -main are pretty fixed at this point, and I've just named the :exec-args after the long-form command-line arguments I believe. Changing them would break backward compatibility so I don't want to do that. Re: deps-deploy -- I think it's pretty important that pom.xml be the source of truth for the group/artifact/version -- at least insofar as the version that is included in the JAR and uploaded to Clojars with the artifact -- so I don't think it's a good idea to override the version in deps-deploy and end up with the internal POM not matching the external POM.#2020-11-0617:27vlaaadAnd then there is me who thinks that pom.xml it's just a temporary file that should be deleted as soon as deployment is done#2020-11-0617:27seancorfieldAt this point, I would prefer to defer all of this to tools.build but I don't know whether deployment is in scope for that (we have local mvn-install already in t.d.a but no remote deployment).#2020-11-0617:27seancorfieldI don't want to have to keep maintaining a community tool if there is an official tool from the core Clojure team that does the same job.#2020-11-0617:28rickmoynihan@seancorfield: yeah that’s fair… I’d much rather overriding the :mvn/version etc happened in depstar. I said this the other day in fact.#2020-11-0617:28alexmillerI think deployment is one of many things that could be invoked by tools.build#2020-11-0617:29alexmillerwhether that's something provided as part of tools.build or community is a separate question. I've looked at it in depth, have a lot of opinions, have not built it.#2020-11-0617:29seancorfield@vlaaad Well, the reality is that there's a lot of stuff in pom.xml that other tooling out there relies on, far beyond the minimum that clojure -Spom produces, so you pretty much have to maintain that information somewhere under version control.#2020-11-0617:32vlaaadOf course, I just think that pom.xml is a horrible storage for this information, since there is a lot of complexity with regards to updating it#2020-11-0617:33seancorfieldOh totally agree! XML is a horrible source format -- but we're "not supposed to use" deps.edn for this stuff, which is where I would otherwise expect it to be... although with :exec-args etc, that's getting more reasonable and "sanctioned" as appropriate by the core team.#2020-11-0617:30seancorfield@rickmoynihan Yeah, there are parts of pom.xml that it would be nice for depstar to be able to override but if you override <version>, you pretty much have to override the SCM <tag> to match, because of tooling that uses that.#2020-11-0617:31rickmoynihanyeah @U09LZR36F mentioned this the other day… that’s trivial to do too though.#2020-11-0617:32seancorfieldA guiding principle for depstar is no additional dependencies so the more functionality it accretes, the more I end up rewriting other tooling into the depstar codebase.#2020-11-0617:35rickmoynihanWhy no additional dependencies? A stable xml lib would be handy, and tools are usually separate from an apps :deps anyway.#2020-11-0617:35rickmoynihaneven tools.deps has dependencies!#2020-11-0617:36rickmoynihanand clojure 🙂#2020-11-0617:37seancorfieldLook at the way depstar works and you'll see: it packages up what it finds on the classpath you used to run it, simply excluding itself.#2020-11-0617:37seancorfieldThat ensures you get exactly what you expect in the JAR based on the aliases you provide, and mean that you don't have to tell depstar itself how to read/combine aliases/`deps.edn` etc.#2020-11-0617:38rickmoynihanIt also allows you to set an explicit :classpath though right?#2020-11-0617:38seancorfieldYes, that's an alternative option but I don't want it to be the default.#2020-11-0617:38seancorfieldLots of people seem to struggle with the other JAR-makers out there and end up at depstar because it is simple and does the obvious, intuitive thing.#2020-11-0617:40rickmoynihansounds more like easy to me… simple would be having the :classpath supplied explicitly.#2020-11-0617:42seancorfieldThe original version of depstar that I inherited had pretty much no options and just created a JAR from the classpath. It was about as simple as it could be. Adding AOT and main-class and other stuff has made it less simple as a tool, but still easy 🙂#2020-11-0617:50rickmoynihanyeah I understand, and there’s nothing wrong with easy.
It just sounds like that’s a requirement for the uberjar side of depstar; but the library side is I think a far simpler problem. Perhaps it should be split into two separate libraries, depstar.jar and depstar.uber?#2020-11-0617:52seancorfieldIt could be refactored to keep those two things separated. Lots of things could be done with depstar but, right now, I'm waiting to see where tools.build ends up.#2020-11-0617:53seancorfieldLike I say, I don't want to keep maintaining a community tool when there's an official core Clojure team tool that does the "same" job.#2020-11-0617:53seancorfieldIf it seems like tools.build won't handle the cases that depstar can handle, then I'll be happy to invest a lot more time into it.#2020-11-0617:54rickmoynihanthat’s fair enough 🙂
but I need solutions now, not in 2+ years 🙂
Being able to build and deploy a library into a private s3 bucket is a common requirement that’s far more awkward than it should be right now.#2020-11-0617:30alexmillerthe code under -Spom has been refactored to make it more amenable to having more options/features and I think longer term that's going to be something we push to primarily being done as part of a build, with probably ever-increasing scope to refine as part of a build#2020-11-0617:30rickmoynihanYeah… For my needs I think treating pom.xml as a template, and moving my override feature from deps-deploy into depstar would make most sense.#2020-11-0617:31alexmillerthere are all sorts of special cases you might want to handle in making a pom for deployment#2020-11-0617:32alexmillermany require some kind of data manipulation you'll want to do during a build#2020-11-0617:33alexmillerjust fyi on tools.build - it exists, it has a lot of functionality but we are trying to finalize a lot of the details around how tasks are defined, stitched together, invoked, etc. b/c that kind of affects everything, we're delaying the public release of it but we'll get there.#2020-11-0617:35seancorfieldAnd I'm still going to bleat on and on about some sort of built-in solution for either the monorepo case or where folks want to combine/insert an extra "master" deps.edn file into the t.d.a mix that everything should then use 🙂#2020-11-0617:35seancorfield(ideally, in t.d.a so that tooling could rely on it easily)#2020-11-0617:36seancorfieldSome sort of "include" or "reference" functionality in deps.edn would work for that -- but I think you've said the core team are opposed to going down that path @alexmiller?#2020-11-0617:37alexmillermy initial lean would be towards an additional deps.edn source but haven't talked about it enough#2020-11-0617:40seancorfieldThat would satisfy me too. I'd like to get away from my current CLJ_CONFIG hack, and I'd like individual developers to be able to keep using their own ~/.clojure/deps.edn file for dev/test stuff without us needing to repro a common core set into our "master" deps.edn file 🙂#2020-11-0617:42rickmoynihanFWIW I think this would be useful too.#2020-11-0618:01rickmoynihan@alexmiller: Which problem does tools.build solve more of? Orchestrating arbitrary tasks together, or providing ways to build jars, deploy stuff etc?
My feeling is that it’s more like make than a way to build jars? Is that right? Or is it both?
I appreciate you don’t want to go into too much detail; but it’d be useful to know if depstar etc might be replaced wholesale by it; or whether something like depstar would just integrate with it?#2020-11-0618:58alexmillerit is both a framework and a set of built-in tasks, so both, and could probably replace a number of existing tools and/or integrate them#2020-11-0618:03rickmoynihanOk sorry folks this is going to have continue another time…
Many thanks to everyone involved in the discussion. I hope you all have fab weekends. 👋 👋#2020-11-0619:01alexmillerI would love to see the built-ins become de facto defaults and make them great with community help#2020-11-0619:41dominicmI'm very curious to see how/if caching is handled. Notoriously difficult task.#2020-11-0619:45alexmillercaching of?#2020-11-0620:10vlaaadI would guess caching of task results#2020-11-0620:11dominicmE.g. make#2020-11-0620:12alexmillerif you mean like compiled classes, that's one thing we're still thinking through#2020-11-0708:34dominicmhttps://github.com/clojure/tools.deps.alpha/blob/d77476f3d5f624249462e275ae62d26da89f320b/src/main/clojure/clojure/tools/deps/alpha/repl.clj#L162-L181 haha, this is great :D#2020-11-0708:36sogaiuyay https://grep.app#2020-11-0714:00alexmillerDon’t get too excited, very experimental#2020-11-0714:04dominicmI just loved the approach :)#2020-11-0714:05rickmoynihanCurrently digging through depstar etc in a spare few minutes, and I’m sat wishing the community had just adopted Stu Halloway’s lancet for builds 12+ years ago 😆
Seriously though, I know ant was a bit of a pain at times but ant as sexps/edn with some deployment stuff would be miles better than what we have right now.#2020-11-0714:06alexmillerI still like many things about ant#2020-11-0714:06rickmoynihanyeah me too#2020-11-0714:07alexmillerThe only thing maven did right was standardize deps and repos. But that was so big, it won.#2020-11-0714:09rickmoynihanYeah was literally about to say just that, and…
The other problem with ant was that every build was different; but it was expressive enough and pretty efficient and the core tasks worked well.
Maven was easier for simple library builds; but far harder for anything that needed to do more than compile, dist, deploy and document.#2020-11-0714:10rickmoynihanYou can integrate resolution and deployment with ant right? Isn’t that what ivy was? I never set that up all those years back.#2020-11-0714:11dominicmGradle seemed very sophisticated when I looked at it.#2020-11-0714:13rickmoynihanI never used it; looked like Rake… essentially ant built as a dsl in groovy right?#2020-11-0714:27borkdudeToday a new version of clj-kondo was released that includes deps.edn linting. You may have to update flycheck-clj-kondo if you are using emacs integration.#2020-11-0714:52alexmillerI have known many people preaching gradle over the years and have given it an honest shot a couple times. I found it inscrutable. It’s probably just me but I just could not understand how to do anything. I’m the same way with ruby though - I just find it baffling. Just give me some parens man.#2020-11-0715:02borkdudeWhat I found most baffling about ruby was rails (I know that's not ruby, but very common): I didn't now where to look or follow things, things just magically came from random places.#2020-11-0715:03borkdudeAt least ruby doesn't have significant whitespace. Don't get me started on that! </rant>#2020-11-0715:04dominicmMy recollection of Gradle was seeing the sheer investment someone had made to integrate ClojureScript with the build system so the caching/dependency system would work.
May not have been Gradle, might have been something else.#2020-11-0717:12flowthingGradle is fine, though quite complex. One big downside it has compared to Ant, IMO, is that it doesn’t have immutable properties.#2020-11-0717:39dharriganWe use gradle a lot at work, more specifically gradle with the kotlin dsl. One can declare vals which are immutable (or vars, which are mutable)#2020-11-0717:40dharriganval foo = "hello" would make foo immutable.#2020-11-0718:41flowthingRight, the Kotlin DSL didn’t exist back when I most used Gradle. Still, I don’t think that helps if you want to define a property in the CLI invocation such that it’s not overwritten somewhere down the chain.#2020-11-0717:40dharriganI think the kotlin dsl is winning (esp for android and spring based projects). Others may still use the groovy dsl.#2020-11-0909:49Eamonn SullivanJust a reminder that I had a practical question ☝️ about how to use an artifactory with ssl client certs authorisation. Our internal artifactory requires -.keyStore and -`http://Djavax.net.keyStorePassword` to be specified. How do I do that for the deps fetching? Is that possible, at the moment?#2020-11-0912:29practicalliSorry, after finding your original question, my previous answer doesnt make sense...#2020-11-0912:56alexmillerThis is not possible right now with clj, but feel free to ask at https://ask.clojure.org and I will turn that into a ticket#2020-11-0915:18Eamonn SullivanDone. Thanks very much. https://ask.clojure.org/index.php/9768/how-can-pass-java-opts-during-dependency-resolution-deps-edn#2020-11-0915:31alexmillerthx#2020-11-0915:31alexmillerthere are some other cases where passing java opts to dep resolution would be useful#2020-11-0909:58borkdude@eamonn.sullivan Maybe not exactly what you are looking for, but if tools.deps doesn't have an answer right now: I've got a port of the clojure CLI script in Clojure itself. It supports setting a bunch of java properties for dep fetching through environment variables. Right now only proxy settings, but your variables may also be appropriate to support.
Take a look at https://github.com/borkdude/deps.clj.#2020-11-0910:00borkdudeYou can run with the JVM, with babashka or as a graal binary. You can even run it as a clojure dep (since it's just normal clojure code) which is kind of meta.#2020-11-0911:15Eamonn SullivanThanks, @borkdude, I will take a look.#2020-11-1103:15nateWhere are all the places you can use an alias keyword to interpolate data from aliases? I've seen examples in :paths and :exec-args, and I'm wondering how far the rabbit hole goes...#2020-11-1103:18alexmillerthose are it right now#2020-11-1103:21natecool#2020-11-1103:21natethank you#2020-11-1109:30henryw374I'm looking for a tool that will help pull out unnecessary :exclusions - ie keeping the set of resolved dependencies unchanged for some combination of aliases. anyone know of such a thing? thanks#2020-11-1110:08borkdude@henryw374 I once diagnosed a dependency problem like this using clojure.spec:
https://github.com/borkdude/dependency-problem#2020-11-1112:09henryw374another thing I'm looking for is a documented description of how deps chooses which version of a dependency to use. I expect it's there but I just can't see it on http://clojure.org and so far google has failed#2020-11-1112:15delaguardohttps://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions.clj#L95-L104
the best place I can think of)
and there is how this defmulti is implemented for different coordinate type
https://github.com/clojure/tools.deps.alpha/search?q=compare-versions#2020-11-1112:27henryw374thanks#2020-11-1113:08alexmillerI did a talk at the conj about it, that’s probably the best description right now. The summary though is that it tries to use the newest version it can.#2020-11-1113:10alexmillerhttps://youtu.be/7CM7Ef-dPWQ#2020-11-1114:46andy.fingerhut"the newest version it can" could be easily misinterpreted as "the newest version available on Maven central and/or Clojars", e.g. similar to behavior of using a -SNAPSHOT or -RELEASE version.#2020-11-1114:47andy.fingerhutWould it be accurate (but slightly longer) to say "the newest version among all of those versions explicitly requested by at least one of your dependencies (but top level can override that)" ?#2020-11-1114:53alexmilleryes#2020-11-1113:09alexmillerBut top level dep versions always win#2020-11-1113:17henryw374gottit! thanks#2020-11-1114:57imreis there any way to tell the clj command to ignore my custom .clojure/deps.edn?#2020-11-1114:58imreah, I may have found it, -Srepro#2020-11-1416:07rickmoynihan@alexmiller: The other week I built a dumb tool to override pom version numbers, but realised that clojure -X:deps mvn-pom almost has this functionality baked in it’s just not exposed.
Is there any reason why tools.deps can’t let you pass [:params :version] here: https://github.com/clojure/tools.deps.alpha/blob/65b1f039929acdd852aae60c87b2e57606e099a3/src/main/clojure/clojure/tools/cli/api.clj#L73
?#2020-11-1416:16alexmillerno real reason - the internals have been refactored to make this feasible but really I expect this to probably go away in favor of doing this via tools.build (which uses most of this same code)#2020-11-1416:18rickmoynihanok in that case my tool wasn’t a complete waste of time 😂#2020-11-1416:33vlaaadAh, I hoped tools-build would have a solution for xml madness, that's great to hear!#2020-11-1416:39seancorfield@rickmoynihan...and this is why I've been pushing back on the depstar discussion until I see what tools.build actually brings to the table: I don't want to do work that would be made redundant by tools.build because I'd much rather use an "official" tool chain, maintained by Nubank, than have to keep maintaining and working with ad hoc third party stuff 🙂#2020-11-1615:33rickmoynihanyup I’ve never questioned that… I’m just trying to improve this area for myself and maybe others in the interim#2020-11-1513:38borkdudeborkdude/babashka {:git/url ""
:sha "41ede17be04965c975e1c0f00e221b02bc5cf879"}
I'm getting:
$ clojure -Sforce -P -A:native
Error building classpath. Manifest type not detected when finding deps for borkdude/babashka in coordinate {:git/url "", :sha "41ede17be04965c975e1c0f00e221b02bc5cf879"}
Why is this? With :deps/manifest :deps added it "works" but it can't find resources in that project#2020-11-1513:40borkdudeMaybe it has issues with projects with git modules?#2020-11-1513:41borkdudeI'll just use the mvn/version one#2020-11-1514:28alexmillerThere is a known issue with git submodules#2020-11-1514:29alexmillerThere’s a ticket for that, not sure if that’s what you’re seeing#2020-11-1515:16borkdudeNo worries, I already solved it a different way#2020-11-1618:43jplazaHi! I’m having this issue https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#long-classpath is there any other workaround besides the {:mvn/local-repo "/path/to/repo"} one? That is not working either#2020-11-1618:52borkdude@jplaza Hmmm. You could spit out the classpath to a file and then start Clojure using java -cp @classpath-file clojure.main but this is a bit spartan maybe.#2020-11-1618:53alexmillerthere is a ticket with some work on it but I don't know that they are workarounds currently#2020-11-1618:54alexmillerhttps://clojure.atlassian.net/browse/TDEPS-120#2020-11-1713:06teroheI tried the @$CpFile suggested and seemed to work. Added a sample diff here: https://github.com/clojure/brew-install/pull/4#issuecomment-728905508
Doesn't check the Java version though so would need something extra for Java 8 (if needed).#2020-11-1618:56borkdude@jplaza
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "3b595449505f4e5f5e7b76796b090b0a02"}, :content ("[email protected]")}#2020-11-1618:57borkdudeWhat this does: java reads the classpath from a file here.#2020-11-1618:57borkdudeTools.deps / clojure CLI might leverage the same workaround#2020-11-1618:57alexmillerseems like it could easily do so since it always has a classpath file already :)#2020-11-1618:58borkdudeyeah#2020-11-1618:59alexmillerthis would actually be trivial#2020-11-1618:59alexmillerif you wanted to hack your ClojureTools.psm1 and try it#2020-11-1619:01alexmillerbasically where you have -classpath $CP if you changed that to "@$CP" (probably not the right syntax but that's the idea) in like line 396#2020-11-1619:02alexmiller$CP here is the name of a classpath file#2020-11-1619:03alexmillerthe exec case is maybe the hardest as it adds the special exec jar: -classpath "$CP;$InstallDir/exec.jar" so might need to emit a well-known variant for that case or something#2020-11-1619:04alexmillerif someone wanted to work on a patch, would be happy to integrate that#2020-11-1619:04borkdudeyou would have to append $InstallDir/exec.jar to the classpath file and then read that. Probably better to append it to some other file classpath-exec and then use that one#2020-11-1619:05borkdudeafaik you can only use one @ and not in combination with non-file classpath#2020-11-1619:06alexmilleryes that would be my expectation. I'm suggesting that right now we write 12341234.cp and we could also write 12341234-exec.cp (which included the exec jar) and use that instead for the -X case#2020-11-1619:06borkdudeyes#2020-11-1619:10borkdudeYou might want to check compatibility of this over several java versions#2020-11-1619:14borkdudeThe java docstring doesn't mention this explicitly#2020-11-1619:16alexmillerbeen around for a long time#2020-11-1619:16alexmillerif it works on Java 8, then it's ok by me :)#2020-11-1619:17borkdude> been around for a long time
where do you find docs about this?#2020-11-1619:18alexmillerlooks like maybe java 9?#2020-11-1619:18borkdudeI'll try java 8#2020-11-1619:19borkdudenope, didn't work#2020-11-1619:19alexmillerdef doc'ed here https://docs.oracle.com/javase/9/tools/java.htm#2020-11-1619:20alexmillerI think maybe it was added in java 9 based on my googling. I thought it was older than that#2020-11-1619:21alexmillerwell, could make it version dependent - if java version is 9+, use @#2020-11-1619:21borkdudeRight, it seems the @argument file thing applies to every option#2020-11-1619:21alexmilleror maybe invert, if Java version is 8, don't use @#2020-11-1619:23borkdudeYeah, that makes sense. That will assume you will not re-use your cache for different java versions, so maybe include the java version in the cache hash if you haven't already#2020-11-1619:24alexmillerI don't think you need that#2020-11-1619:25alexmillerthe files are the same either way, it's just whether or not you can use it via @#2020-11-1619:25borkduderight. how will the CLI know which java version is used?#2020-11-1619:25alexmillerby calling java -version ?#2020-11-1619:26borkdude$ time java -version
java -version 0.07s user 0.06s system 94% cpu 0.136 total
#2020-11-1619:27alexmillerwhat's the error if you use the @file on java 8?#2020-11-1619:27borkdude$ java -cp @classpath clojure.main
Error: Could not find or load main class clojure.main
#2020-11-1619:27alexmillernot too distinguishable then#2020-11-1619:28borkdudeMaybe this can be controlled via an environment variable#2020-11-1619:28alexmillerI was wondering if you could just assume it would work, detect the failure and retry#2020-11-1619:28borkdudeCLOJURE_CP_FILE=true#2020-11-1619:28alexmilleryeah, it could be an env var too. I was trying to avoid that :)#2020-11-1619:28alexmillermight be the cleanest though#2020-11-1619:29borkdudefor those experiencing this problem, they would set it, others won't#2020-11-1619:29alexmillerthat's fine#2020-11-1619:29borkdudeand only on Java 9+.#2020-11-1619:30borkdude@jplaza what Java version are you using?#2020-11-1619:32borkdude@alexmiller alternative dirty workaround: use add-lib to load the libs later on ;)#2020-11-1619:34alexmillerbleh#2020-11-1619:34borkdudeyeah, I said dirty, but it's always good to list alternatives, even when they're bad ;)#2020-11-1619:36borkdudeanother alternative: won't fix until Java 8 isn't supported anymore :)#2020-11-1619:36alexmillerwho knows if that day will ever come#2020-11-1619:37borkdudeanother alternative: reverse engineer the java command line tool and build a nicer front-end which supports this for older java versions ;)#2020-11-1619:41jplazaThanks for the fast reply @alexmiller @borkdude! I’m helping a teammate with that issue. She has java 9.#2020-11-1619:42jplazaI told her the problem is that she’s using Windows haha! She’s installing Windows Subsytem for Linux#2020-11-1619:42borkdudeas a quick workaround: https://clojurians.slack.com/archives/C6QH853H8/p1605552978403300#2020-11-1619:43borkdude#2020-11-1619:46jplaza😆#2020-11-1619:47borkdude> Windows command-line length limit
Does this apply to cmd.exe, powershell.exe or is it a conhost limit? Do you experience the same problem with the new Windows Terminal?#2020-11-1619:48jplazaNot sure but happens with both cmd and powershell#2020-11-1619:48borkdudewsl2 is pretty good ;)#2020-11-1619:51seancorfieldYeah, I gave up on the whole Powershell thing and just do everything on WSL2 now. Esp. with VS Code's Remote-WSL2 extension.#2020-11-1619:53borkdudeHave you also tried portal btw?#2020-11-1620:01seancorfieldYes, but I prefer to dock a GUI next to my editor rather than switch back and forth between an editor and a browser.#2020-11-1619:52seancorfield(I run VcXsrv aka Xlaunch so I can start X11-based GUIs from WSL2, such as REBL or Reveal)#2020-11-1620:51jplazaCool! Any tips on using Calva + WSL2?#2020-11-1620:53seancorfield@jplaza I use Clover with VS Code. I like working with a plain Socket REPL in processes -- I avoid nREPL and all that stuff. Clover is "Chlorine for VS Code" and I was previously using Chlorine on Atom.#2020-11-1620:59jplazaOK I will try that. I already cloned your deps.edn https://github.com/seancorfield/dot-clojure to play with the sock repl#2020-11-1621:00borkdude@jplaza Many people use Calva. Check out #calva. There's a lot going on there.#2020-11-1621:01borkdude@jplaza Also check out out the clj-kondo plugin for VScode </shameless-plug>#2020-11-1622:34jplazaWe use clj-kondo in our team! Awesome tool! I actually mentioned our startup in the issue so you can list us there 😉#2020-11-1621:32gmorphemeI assumed clojure -X:deps mvn-pom :lib 'foo/bar' :version '"0.0.1-SNAPSHOT"' would generate generate me a POM with the specified details but I’m not having any luck - I just get defaults. (Maybe mvn-pom doesn’t pass the args on through to sync-pom?). Was this not an intended usage? Or am I missing something? Or just an oversight?#2020-11-1621:57alexmillerit does not pass those through right now#2020-11-1621:57alexmillermvn-pom may go away in favor of tools.build once that is out so I'm resisting the urge to make it better#2020-11-1709:11gmorphemeunderstood, thanks#2020-11-1709:27rickmoynihanIn the mean time you might try this:
https://github.com/rickmoynihan/pom-update#2020-11-1709:29rickmoynihanhot off the press so YMMV, I’ve not even properly announced it yet.#2020-11-1709:41gmorphemeThanks - works for me if I generate the POM with tools.deps then update the version with pom-update. I needed to add :pom/write-strategy :overwrite :pom/input-file '"pom.xml"' explicitly though.#2020-11-1711:02rickmoynihanoh yes thanks that’s intentional, I just forgot to add that to the README.md when I wrote it up last night.
I’ll do that now.
Thanks for letting me know.#2020-11-1711:03rickmoynihanFixed.#2020-11-1711:03rickmoynihanMy recommendation is that you include those in your deps.edn rather than via command line, as they won’t vary across calls.#2020-11-1711:03rickmoynihanSee the updated README.md#2020-11-1622:12pez@jplaza if you find out stuff about using WSL2 with Calva, please consider updating this page: https://calva.io/wsl/#2020-11-1622:35jplazaWill do #2020-11-1714:53imreHey folks. I just ran into a problem at deps resolution: Error building classpath. Could not find artifact gcardone:junidecode:jar:0.2 in central ()
My project is not directly referencing this lib and I don't know where in the dep tree it is referenced from. Is there any way to trace this with the clojure command line?#2020-11-1714:55borkdudeclojure -Stree#2020-11-1714:55imreUnfortunately that fails with the same error without printing anything useful. So do -Spath and -Strace#2020-11-1714:56borkdudeoh right :)#2020-11-1714:56borkdudethen bisect?#2020-11-1714:57borkdudemaybe also try -Sforce which will force recreation of classpath#2020-11-1714:57imrebisect it is then#2020-11-1714:57imre😞#2020-11-1714:57imreforce does not help#2020-11-1714:59dominicmI'm surprised trace doesn't output anything. That would be pretty useful for this case.#2020-11-1714:59dominicmIs the secret verbose flag still a thing the?#2020-11-1715:00alexmillerno#2020-11-1715:00alexmillerthere isn't a good answer for this right now other than tracing down the deps tree semi-manually#2020-11-1715:00alexmillerI've been looking at some new tools for questions like this#2020-11-1715:01alexmilleror you could -Spom and mvn dependency:tree#2020-11-1715:01alexmillermight get more info that way#2020-11-1715:01imrehmm could try that yeah#2020-11-1715:01imrewell, that also fails with the same I'm afraid#2020-11-1715:02alexmillermaybe mvn -X dependency:tree ?#2020-11-1715:03imreSorry, I meant -Spom fails#2020-11-1715:04alexmilleroh, right :)#2020-11-1715:21imrefound it in the end by reverting and updating deps one by one#2020-11-1915:24Michaël SalihiHi!#2020-11-1915:26Michaël SalihiIs it possible to use :local/root with a local Java library ?#2020-11-1915:31alexmilleronly if you independently compile the .java files into .class and include the class dir as a source path#2020-11-1915:37Michaël SalihiPerfect, that's what I thought.
Thanks#2020-11-2014:05millettjonin deps.edn, is it possible to have one alias include another?#2020-11-2014:10dominicm@millettjon You can have an alias include directories from another like so:
:aliases {:foo-paths ["src"] :some-alias {:extra-paths [:foo-paths "some-alias-path"]}}
#2020-11-2014:14delaguardoare you sure?
clj -Sdeps '{:paths ["foo"] :aliases {:foo {:extra-paths ["bar"]} :bar {:extra-paths [:foo "baz"]}}}' -A:bar -Spath
=> baz:foo:...
without paths from alias :foo#2020-11-2014:16millettjonLooks like this is the JIRA ticket: https://clojure.atlassian.net/browse/TDEPS-170#2020-11-2014:16dominicm@U04V4KLKC You changed my example :)#2020-11-2014:19millettjonThanks. I would use both in this case.#2020-11-2014:24delaguardo@U09LZR36F right, I didn’t know that you can have a vector of strings as a value for alias keyword. Not sure if this is a part of official api thought#2020-11-2014:30borkdudeHuh, is this new?#2020-11-2014:31borkdudeAnd does it work or not? I see conflicting evidence with @U04V4KLKC's example?#2020-11-2014:34delaguardo:aliases {:foo ["X"]
:bar {:extra-paths [:foo "Y"]}
is working
clj -Sdeps '{:aliases {:foo ["X"] :bar {:extra-paths [:foo "Y"]}}}' -Srepro -A:bar -Spath
=> X:Y:...
but it also is not valid acording the spec in tools.deps.alpha — https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/specs.clj#2020-11-2014:34millettjonI can see the jira ticket but don't see how to vote. Maybe I don't have the right access?#2020-11-2014:35millettjonYeah I didn't see it mentioned anywhere but this is the second time I have wanted to use include functionality.#2020-11-2014:35borkdudeThat is a common issue with JIRA. It sucks.#2020-11-2015:14dominicm@U04V4KLKC this is part of the official API, yes.#2020-11-2015:14dominicmThis is new in tools deps#2020-11-2015:19delaguardodo you know where this declared? Can’t find it in cli guide or reference#2020-11-2015:23delaguardoalso (if this is new) when it was introduced?#2020-11-2015:24delaguardonwm, found it in changelog#2020-11-2015:25borkdudewhen?#2020-11-2015:26delaguardoin 0.9.745#2020-11-2015:26delaguardo• Added support for using data stored in aliases as :paths#2020-11-2015:26borkdudedo you have a link to the changelogs? I never know where to look: in brew install clojure or tools-deps?#2020-11-2015:26delaguardohttps://github.com/clojure/tools.deps.alpha/blob/master/CHANGELOG.md#2020-11-2015:29delaguardobut this is soooo confusing
clj -Sdeps '{:aliases {:foo ["x"]}}' -A:foo -Spath
=> "src:..."
seems like this isn’t working in that case#2020-11-2015:29borkdudeI would have never guessed that
> Added support for using data stored in aliases as :paths
this would mean the above#2020-11-2015:29delaguardosame#2020-11-2016:49seancorfield@U04V4KLKC There's an example in the docs of this https://clojure.org/reference/deps_and_cli#_paths#2020-11-2016:50seancorfield{:paths [:clj-paths :resource-paths]
:aliases
{:clj-paths ["src/clj" "src/cljc"]
:resource-paths ["resources"]}}#2020-11-2016:50seancorfieldHere are the CLI release notes: https://clojure.org/releases/tools#2020-11-2016:52seancorfieldAh, I see Alex linked you to the same example in the docs (but at least the CLI releases link is new to this thread).#2020-11-2016:53delaguardoyes, this one is new, thanks!#2020-11-2016:54seancorfieldYou can also use aliases for :exec-args to point to a data structure that is then used as the arguments to the :exec-fn#2020-11-2016:55delaguardoas a complete replacement of exec-args? like :exec-args :one-of-aliases ?#2020-11-2016:55seancorfieldSo far those are the only two places where the CLI/t.d.a itself allows/uses aliased data. Yes, like that.#2020-11-2016:56seancorfield(the :exec-args usage is not documented on that page, unfortunately)#2020-11-2016:57seancorfieldWith the Clojure basis available to any program run from the CLI, you can also read the merged deps.edn data and access aliased data inside your program at runtime.#2020-11-2017:00delaguardoI wish to have ability to provide custom data literals for deps.edn reader. then those tiny references to data in aliases can be colored as #aliases/ref :some-alias and allows to be used as a partial replacement for exec-args#2020-11-2017:01delaguardowithout this twick it is hard to say what is going on here :paths [:foo "bar"] without reading documentation (with blanks 😞 )#2020-11-2017:04seancorfieldFeel free to post requests to improve the docs on http://ask.clojure.org -- or if you've signed the Contributor's Agreement, you could submit PRs directly to https://github.com/clojure/clojure-site/pulls (but probably best to discuss with @alexmiller first to see how he'd like this clarified). Suggestions for changing the deps.edn file format itself can go on http://ask.clojure.org#2020-11-2014:10dominicmFor including dependencies, there's a JIRA for that and you might be able to vote on it there if you have an account.#2020-11-2014:22jmayaalvis there a way to tell cli where to search for maven settings.xml ? maven allows to override the location with -s (ex: mvn -s myptah/settings.xml install)#2020-11-2015:43alexmillerno#2020-11-2015:43alexmillerif you want to file an https://ask.clojure.org request for that, I don't think we have any such question/ticket right now#2020-11-2015:48jmayaalvWill do ! thanks @alexmiller#2020-11-2015:54delaguardoI have a question -
should this clj -Sdeps '{:aliases {:foo ["x"]}}' -A:foo -Spath
work the same as clj -Sdeps '{:aliases {:foo {:extra-paths ["x"]}}}' -A:foo -Spath ?
right now I’m getting different results. In first case there is no “x” in my path#2020-11-2016:03alexmillerthe first is not a thing#2020-11-2016:03alexmillernot sure what you're expecting there?#2020-11-2016:04alexmillerunless you do
clj -Sdeps '{:paths [:foo] :aliases {:foo ["x"]}}' -A:foo -Spath#2020-11-2016:04delaguardojust curiosity
as an example this clj -Sdeps '{:aliases {:foo ["x"] :bar {:extra-paths [:foo]}}}' -A:bar -Spath is working wierdly#2020-11-2016:04alexmilleras it should#2020-11-2016:06delaguardothat is what I found today in the slack thread
was surprised because can’t remember mention of that in cli guide or reference
also specs in tools.deps.alpha is not allowing such aliases neither as in API.md#2020-11-2016:08alexmillerthere is an example in https://clojure.org/reference/deps_and_cli#_paths but may be under-doc'ed :)#2020-11-2016:09alexmillerspecs may be lagging#2020-11-2016:10delaguardolooks like :aliases become a little bit overloaded key#2020-11-2016:20alexmillernot overloaded at all, just very generic. an alias is a name for a data structure.#2020-11-2016:23delaguardoand this data now is context specific + depend on it’s shape, isn’t it?#2020-11-2016:26alexmillerdependent on how you want to use it, it's just data#2020-11-2016:30delaguardosure, but this clj -Sdeps '{:aliases {:foo "x"}}' -A:foo -Spath
will throw
clj -Sdeps '{:aliases {:foo "x"}}' -A:foo -Spath
Error building classpath. nth not supported on this type: Character
java.lang.UnsupportedOperationException: nth not supported on this type: Character
this exception is telling me that data is used in a context hidden from me. So for me this is not just data#2020-11-2016:31alexmillerwell, I'd say that's a bug#2020-11-2016:32alexmillershould either be ignored or have a better error in that case, feel free to log at http://ask.clojure.org#2020-11-2016:35delaguardosure, I will report it
btw, this clj -Sdeps '{:paths [:foo] :aliases {:foo "x"}}' -Spath
doesn’t give me an exception#2020-11-2016:36alexmilleriirc, it flattens so it's little loosey goosey about the colls there atm, still deciding what to allow#2020-11-2016:43schmuddeI have an issue bringing in a .jar that relies on a pom.xml that relies on Dropwizard. Having a little trouble figuring out where the issue really lies. clj -Spath gives me this error for all the .jar's dependencies: [ERROR] Failed to determine Java version for profile....
I can run the jar just fine with plain old Java. It only becomes a problem when I try to bring it into my project. Here it is in my deps.edn: com.plaid/quickstart {:local/root "src/usermanager/java/target/quickstart-1.0-SNAPSHOT.jar"}#2020-11-2016:44schmuddeThe docs tell me this is the right behavior ("If the jar includes a pom.xml file, it will be used to find transitive deps"). FWIW, each dependency is enumerated thusly: <groupId>io.dropwizard</groupId>, <artifactId>dropwizard-bom</artifactId>, <version>${dropwizard.version}</version>. With the version of Dropwizard resolving as 1.3.2. That what it seems to be choking on, based on the error.#2020-11-2016:45schmuddeShould I just build my own uberjar?#2020-11-2016:47alexmillerresolving pom files is kind of a pain in the ass (super poms, variables, env vars, etc) - it is certainly possible for the pom inside a jar to be insufficient to actually resolve the dep set#2020-11-2016:49schmuddeWell, I have the source. So I can roll it again. Maybe that's the easiest way? this .jar came in the repository.#2020-11-2016:49alexmillerif you have a concrete repro, would be happy to see a question on https://ask.clojure.org with that so I can look at it. at the very least, would like to fail gracefully and let you supply those deps explicitly.#2020-11-2016:50schmuddeDebugging it from the clojure side seems pretty silly if it's not immediately obvious what's going wrong.#2020-11-2016:50schmuddeI do. It's Plaid's API Java code... so it's also something that isn't too obscure.#2020-11-2016:51alexmiller"Failed to determine Java version for profile" means that there is a maven profile that uses maven's Java version stuff (lets you select per-java version deps)#2020-11-2016:51schmuddeI can put the question there. I'll have to give a bit of a think of how to phrase it succinctly.#2020-11-2016:51alexmillerI think maybe that requires JAVA_HOME env var#2020-11-2016:52alexmilleryou could try setting that and seeing if it works#2020-11-2016:52schmuddeGot it. I'll play a little more and post something on http://ask.clojure.org. Thanks @alexmiller!#2020-11-2111:55mishahowdy!
https://akovantsev.github.io/corpus/clojure-slack/tools-deps
this channel's log as a single page (as preserved by https://clojurians-log.clojureverse.org/tools-deps)
double-click on words to filter (cmd+f to highlight as per your browser), click on dates to expand filtered-out.#2020-11-2112:57borkdudenice work. fwiw, it's also here: https://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/tools-deps#2020-11-2115:18misha> Log in to Zulip
opieop#2020-11-2113:58borkdudeWe are using CLJ_CONFIG to set dependency versions properly from a "one source of truth" deps.edn file that is located in ../base.
It works great. Our only problem is generating the pom file:
$ export CLJ_CONFIG=../base
clojure -A:base -X:deps mvn-pom :lib dre.annotator/annotator
Error generating pom manifest: Bad coordinate for library selmer/selmer, expected map: nil
#2020-11-2113:59borkdudeshould we be using -X:base:deps ? this also doesn't work#2020-11-2114:33borkdudeWe're using the :default-deps feature#2020-11-2114:34borkdudeRepro:
{:aliases {:versions {:default-deps {clj-kondo/clj-kondo {:mvn/version "2020.10.10"}}}}
:deps {clj-kondo/clj-kondo nil}}
clojure -A:versions -Spath
works, but:
$ clojure -A:versions -X:deps mvn-pom
Error generating pom manifest: Bad coordinate for library clj-kondo/clj-kondo, expected map: nil
#2020-11-2114:55borkdudemy team lead is now threatening with migrating everything to mvn ;)#2020-11-2114:58borkdudeProbably this is too strict:
https://github.com/clojure/tools.deps.alpha/blob/9b3744fc9ab3b64d1e39765afc6e478c7c77cea4/src/main/clojure/clojure/tools/deps/alpha/extensions.clj#L66-L72
The map? check should be extended to allowing nil as well#2020-11-2118:16seancorfield@borkdude use -Spom instead#2020-11-2118:16seancorfieldWe have the exact same problem at work#2020-11-2118:18seancorfield(also we use {} instead of nil for overridden deps - and then :override-deps#2020-11-2118:40borkdudelol, I made this hack:
clojure -M:base:deps -e "(require '[clojure.tools.cli.api :as api]) (require '[clojure.tools.deps.alpha.extensions :as ext]) (alter-var-root #'ext/throw-bad-coord (constantly (fn [x y] [x y]))) (alter-var-root #'ext/throw-bad-manifest (constantly (fn [x y z] z))) (api/mvn-pom {:lib 'dre.annotator/annotator})"
but it didn't work. I'll try -Spom .#2020-11-2118:41borkdude@seancorfield I'm seeing:
Skipping paths: resources
Skipping coordinate: {:local/root ../genseng, :deps/manifest :deps, :deps/root ../genseng, :parents #{[]}, :paths [/Users/borkdude/dre/DocSearch/genseng/src]}#2020-11-2118:41borkdudewe have a couple of local/root deps which it seems it doesn't pick up?#2020-11-2118:42borkdudeNot sure if this is bad. We use the pom.xml for the uberjar with depstar:
clojure -A:base:jvm -M:depstar -m hf.depstar.uberjar target/annotator-standalone.jar -C -m dre.standalone
#2020-11-2119:40seancorfieldThe pom.xml isn't terribly important to depstar -- it gets artifact, group, and version from it. That's all.#2020-11-2119:43seancorfieldWe generated a pom.xml for each subproject that we build uberjars for but the dependencies aren't important. Then we do the CLJ_CONFIG=../versions clojure -M:defaults:depstar ...etc... much like you @borkdude#2020-11-2119:44borkdudeyep, same. thanks!#2020-11-2119:44borkdudeIn the olden boot days I wrote my own solution for this: https://github.com/borkdude/boot-bundle